Raised when Ruby can’t yield as requested.
A typical scenario is attempting to yield when no block is given:
def call_block yield 42 end call_block
raises the exception:
LocalJumpError: no block given (yield)
A more subtle example:
def get_me_a_return Proc.new { return 42 } end get_me_a_return.call
raises the exception:
LocalJumpError: unexpected return
Raised in case of a stack overflow.
def me_myself_and_i me_myself_and_i end me_myself_and_i
raises the exception:
SystemStackError: stack level too deep
Ractor
is an Actor-model abstraction for Ruby that provides thread-safe parallel execution.
Ractor.new
can make a new Ractor
, and it will run in parallel.
# The simplest ractor r = Ractor.new {puts "I am in Ractor!"} r.take # wait for it to finish # here "I am in Ractor!" would be printed
Ractors do not share usual objects, so the same kinds of thread-safety concerns such as data-race, race-conditions are not available on multi-ractor programming.
To achieve this, ractors severely limit object sharing between different ractors. For example, unlike threads, ractors can’t access each other’s objects, nor any objects through variables of the outer scope.
a = 1 r = Ractor.new {puts "I am in Ractor! a=#{a}"} # fails immediately with # ArgumentError (can not isolate a Proc because it accesses outer variables (a).)
On CRuby (the default implementation), Global Virtual Machine Lock (GVL) is held per ractor, so ractors are performed in parallel without locking each other.
Instead of accessing the shared state, the objects should be passed to and from ractors via sending and receiving objects as messages.
a = 1 r = Ractor.new do a_in_ractor = receive # receive blocks till somebody will pass message puts "I am in Ractor! a=#{a_in_ractor}" end r.send(a) # pass it r.take # here "I am in Ractor! a=1" would be printed
There are two pairs of methods for sending/receiving messages:
Ractor#send
and Ractor.receive
for when the sender knows the receiver (push);
Ractor.yield
and Ractor#take
for when the receiver knows the sender (pull);
In addition to that, an argument to Ractor.new
would be passed to block and available there as if received by Ractor.receive
, and the last block value would be sent outside of the ractor as if sent by Ractor.yield
.
A little demonstration on a classic ping-pong:
server = Ractor.new do puts "Server starts: #{self.inspect}" puts "Server sends: ping" Ractor.yield 'ping' # The server doesn't know the receiver and sends to whoever interested received = Ractor.receive # The server doesn't know the sender and receives from whoever sent puts "Server received: #{received}" end client = Ractor.new(server) do |srv| # The server is sent inside client, and available as srv puts "Client starts: #{self.inspect}" received = srv.take # The Client takes a message specifically from the server puts "Client received from " \ "#{srv.inspect}: #{received}" puts "Client sends to " \ "#{srv.inspect}: pong" srv.send 'pong' # The client sends a message specifically to the server end [client, server].each(&:take) # Wait till they both finish
This will output:
Server starts: #<Ractor:#2 test.rb:1 running> Server sends: ping Client starts: #<Ractor:#3 test.rb:8 running> Client received from #<Ractor:#2 rac.rb:1 blocking>: ping Client sends to #<Ractor:#2 rac.rb:1 blocking>: pong Server received: pong
It is said that Ractor
receives messages via the incoming port, and sends them to the outgoing port. Either one can be disabled with Ractor#close_incoming
and Ractor#close_outgoing
respectively. If a ractor terminated, its ports will be closed automatically.
When the object is sent to and from the ractor, it is important to understand whether the object is shareable or unshareable. Most of objects are unshareable objects.
Shareable objects are basically those which can be used by several threads without compromising thread-safety; e.g. immutable ones. Ractor.shareable?
allows to check this, and Ractor.make_shareable
tries to make object shareable if it is not.
Ractor.shareable?(1) #=> true -- numbers and other immutable basic values are Ractor.shareable?('foo') #=> false, unless the string is frozen due to # freeze_string_literals: true Ractor.shareable?('foo'.freeze) #=> true ary = ['hello', 'world'] ary.frozen? #=> false ary[0].frozen? #=> false Ractor.make_shareable(ary) ary.frozen? #=> true ary[0].frozen? #=> true ary[1].frozen? #=> true
When a shareable object is sent (via send
or Ractor.yield
), no additional processing happens, and it just becomes usable by both ractors. When an unshareable object is sent, it can be either copied or moved. The first is the default, and it makes the object’s full copy by deep cloning of non-shareable parts of its structure.
data = ['foo', 'bar'.freeze] r = Ractor.new do data2 = Ractor.receive puts "In ractor: #{data2.object_id}, #{data2[0].object_id}, #{data2[1].object_id}" end r.send(data) r.take puts "Outside : #{data.object_id}, #{data[0].object_id}, #{data[1].object_id}"
This will output:
In ractor: 340, 360, 320 Outside : 380, 400, 320
(Note that object id of both array and non-frozen string inside array have changed inside the ractor, showing it is different objects. But the second array’s element, which is a shareable frozen string, has the same object_id.)
Deep cloning of the objects may be slow, and sometimes impossible. Alternatively, move: true
may be used on sending. This will move the object to the receiving ractor, making it inaccessible for a sending ractor.
data = ['foo', 'bar'] r = Ractor.new do data_in_ractor = Ractor.receive puts "In ractor: #{data_in_ractor.object_id}, #{data_in_ractor[0].object_id}" end r.send(data, move: true) r.take puts "Outside: moved? #{Ractor::MovedObject === data}" puts "Outside: #{data.inspect}"
This will output:
In ractor: 100, 120 Outside: moved? true test.rb:9:in `method_missing': can not send any methods to a moved object (Ractor::MovedError)
Notice that even inspect
(and more basic methods like __id__
) is inaccessible on a moved object.
Besides frozen objects, there are shareable objects. Class
and Module
objects are shareable so the Class/Module definitions are shared between ractors. Ractor
objects are also shareable objects. All operations for the shareable mutable objects are thread-safe, so the thread-safety property will be kept. We can not define mutable shareable objects in Ruby, but C extensions can introduce them.
It is prohibited to access instance variables of mutable shareable objects (especially Modules and classes) from ractors other than main:
class C class << self attr_accessor :tricky end end C.tricky = 'test' r = Ractor.new(C) do |cls| puts "I see #{cls}" puts "I can't see #{cls.tricky}" end r.take # I see C # can not access instance variables of classes/modules from non-main Ractors (RuntimeError)
Ractors can access constants if they are shareable. The main Ractor
is the only one that can access non-shareable constants.
GOOD = 'good'.freeze BAD = 'bad' r = Ractor.new do puts "GOOD=#{GOOD}" puts "BAD=#{BAD}" end r.take # GOOD=good # can not access non-shareable objects in constant Object::BAD by non-main Ractor. (NameError) # Consider the same C class from above r = Ractor.new do puts "I see #{C}" puts "I can't see #{C.tricky}" end r.take # I see C # can not access instance variables of classes/modules from non-main Ractors (RuntimeError)
See also the description of shareable_constant_value pragma in Comments syntax explanation.
Each ractor creates its own thread. New threads can be created from inside ractor (and, on CRuby, sharing GVL with other threads of this ractor).
r = Ractor.new do a = 1 Thread.new {puts "Thread in ractor: a=#{a}"}.join end r.take # Here "Thread in ractor: a=1" will be printed
In examples below, sometimes we use the following method to wait till ractors that are not currently blocked will finish (or process till next blocking) method.
def wait sleep(0.1) end
It is **only for demonstration purposes** and shouldn’t be used in a real code. Most of the times, just take
is used to wait till ractor will finish.
See Ractor design doc for more details.
Random
provides an interface to Ruby’s pseudo-random number generator, or PRNG. The PRNG produces a deterministic sequence of bits which approximate true randomness. The sequence may be represented by integers, floats, or binary strings.
The generator may be initialized with either a system-generated or user-supplied seed value by using Random.srand
.
The class method Random.rand
provides the base functionality of Kernel.rand
along with better handling of floating point values. These are both interfaces to the Ruby system PRNG.
Random.new
will create a new PRNG with a state independent of the Ruby system PRNG, allowing multiple generators with different seed values or sequence positions to exist simultaneously. Random
objects can be marshaled, allowing sequences to be saved and resumed.
PRNGs are currently implemented as a modified Mersenne Twister with a period of 2**19937-1. As this algorithm is not for cryptographical use, you must use SecureRandom
for security purpose, instead of this PRNG.
See also Random::Formatter
module that adds convenience methods to generate various forms of random data.
Raised when given an invalid regexp expression.
Regexp.new("?")
raises the exception:
RegexpError: target of repeat operator is not specified: /?/
Raised when an invalid operation is attempted on a thread.
For example, when no other thread has been started:
Thread.stop
This will raises the following exception:
ThreadError: stopping only thread note: use sleep to stop forever
The exception class which will be raised when pushing into a closed Queue. See Thread::Queue#close
and Thread::SizedQueue#close
.
Document-class: TracePoint
A class that provides the functionality of Kernel#set_trace_func
in a nice Object-Oriented API.
We can use TracePoint
to gather information specifically for exceptions:
trace = TracePoint.new(:raise) do |tp| p [tp.lineno, tp.event, tp.raised_exception] end #=> #<TracePoint:disabled> trace.enable #=> false 0 / 0 #=> [5, :raise, #<ZeroDivisionError: divided by 0>]
If you don’t specify the type of events you want to listen for, TracePoint
will include all available events.
Note do not depend on current event set, as this list is subject to change. Instead, it is recommended you specify the type of events you want to use.
To filter what is traced, you can pass any of the following as events
:
:line
execute an expression or statement on a new line
:class
start a class or module definition
:end
finish a class or module definition
:call
call a Ruby method
:return
return from a Ruby method
:c_call
call a C-language routine
:c_return
return from a C-language routine
:raise
raise an exception
:b_call
event hook at block entry
:b_return
event hook at block ending
:a_call
event hook at all calls (call
, b_call
, and c_call
)
:a_return
event hook at all returns (return
, b_return
, and c_return
)
:thread_begin
event hook at thread beginning
:thread_end
event hook at thread ending
:fiber_switch
event hook at fiber switch
:script_compiled
new Ruby code compiled (with eval
, load
or require
)
Raised when throw
is called with a tag which does not have corresponding catch
block.
throw "foo", "bar"
raises the exception:
UncaughtThrowError: uncaught throw "foo"
RubyGems adds the gem
method to allow activation of specific gem versions and overrides the require
method on Kernel
to make gems appear as if they live on the $LOAD_PATH
. See the documentation of these methods for further detail.
Monkey patch kernel to ensure that all ‘require` calls call the same method
The Kernel
module is included by class Object
, so its methods are available in every Ruby object.
The Kernel
instance methods are documented in class Object
while the module methods are documented here. These methods are called without a receiver and thus can be called in functional form:
sprintf "%.1f", 1.234 #=> "1.2"
Module Kernel provides methods that are useful for:
__callee__
: Returns the called name of the current method as a symbol.
__dir__
: Returns the path to the directory from which the current method is called.
__method__
: Returns the name of the current method as a symbol.
autoload?
: Returns the file to be loaded when the given module is referenced.
binding
: Returns a Binding
for the context at the point of call.
block_given?
: Returns true
if a block was passed to the calling method.
caller
: Returns the current execution stack as an array of strings.
caller_locations
: Returns the current execution stack as an array of Thread::Backtrace::Location
objects.
class
: Returns the class of self
.
frozen?
: Returns whether self
is frozen.
global_variables
: Returns an array of global variables as symbols.
local_variables
: Returns an array of local variables as symbols.
test
: Performs specified tests on the given single file or pair of files.
abort
: Exits the current process after printing the given arguments.
at_exit
: Executes the given block when the process exits.
exit
: Exits the current process after calling any registered at_exit
handlers.
exit!
: Exits the current process without calling any registered at_exit
handlers.
catch
: Executes the given block, possibly catching a thrown object.
raise
(aliased as fail
): Raises an exception based on the given arguments.
throw
: Returns from the active catch block waiting for the given tag.
::pp
: Prints the given objects in pretty form.
gets
: Returns and assigns to $_
the next line from the current input.
open
: Creates an IO
object connected to the given stream, file, or subprocess.
p
: Prints the given objects’ inspect output to the standard output.
print
: Prints the given objects to standard output without a newline.
printf
: Prints the string resulting from applying the given format string to any additional arguments.
putc
: Equivalent to <tt.$stdout.putc(object)</tt> for the given object.
puts
: Equivalent to $stdout.puts(*objects)
for the given objects.
readline
: Similar to gets
, but raises an exception at the end of file.
readlines
: Returns an array of the remaining lines from the current input.
lambda
: Returns a lambda proc for the given block.
set_trace_func
: Sets the given proc as the handler for tracing, or disables tracing if given nil
.
trace_var
: Starts tracing assignments to the given global variable.
untrace_var
: Disables tracing of assignments to the given global variable.
`command`: Returns the standard output of running command
in a subshell.
exec
: Replaces current process with a new process.
fork
: Forks the current process into two processes.
spawn
: Executes the given command and returns its pid without waiting for completion.
system
: Executes the given command in a subshell.
autoload
: Registers the given file to be loaded when the given constant is first referenced.
load
: Loads the given Ruby file.
require
: Loads the given Ruby file unless it has already been loaded.
require_relative
: Loads the Ruby file path relative to the calling file, unless it has already been loaded.
tap
: Yields self
to the given block; returns self
.
then
(aliased as yield_self
): Yields self
to the block and returns the result of the block.
rand
: Returns a pseudo-random floating point number strictly between 0.0 and 1.0.
srand
: Seeds the pseudo-random number generator with the given number.
eval
: Evaluates the given string as Ruby code.
loop
: Repeatedly executes the given block.
sleep
: Suspends the current thread for the given number of seconds.
sprintf
(aliased as format
): Returns the string resulting from applying the given format string to any additional arguments.
syscall
: Runs an operating system call.
trap
: Specifies the handling of system signals.
warn
: Issue a warning based on the given messages and options.
Module Enumerable provides methods that are useful to a collection class for:
These methods return information about the Enumerable other than the elements themselves:
include?
, member?
: Returns true
if self == object
, false
otherwise.
all?
: Returns true
if all elements meet a specified criterion; false
otherwise.
any?
: Returns true
if any element meets a specified criterion; false
otherwise.
none?
: Returns true
if no element meets a specified criterion; false
otherwise.
one?
: Returns true
if exactly one element meets a specified criterion; false
otherwise.
count
: Returns the count of elements, based on an argument or block criterion, if given.
tally
: Returns a new Hash containing the counts of occurrences of each element.
These methods return entries from the Enumerable, without modifying it:
Leading, trailing, or all elements:
first
: Returns the first element or leading elements.
take
: Returns a specified number of leading elements.
drop
: Returns a specified number of trailing elements.
take_while
: Returns leading elements as specified by the given block.
drop_while
: Returns trailing elements as specified by the given block.
Minimum and maximum value elements:
min
: Returns the elements whose values are smallest among the elements, as determined by <=>
or a given block.
max
: Returns the elements whose values are largest among the elements, as determined by <=>
or a given block.
minmax
: Returns a 2-element Array containing the smallest and largest elements.
min_by
: Returns the smallest element, as determined by the given block.
max_by
: Returns the largest element, as determined by the given block.
minmax_by
: Returns the smallest and largest elements, as determined by the given block.
Groups, slices, and partitions:
group_by
: Returns a Hash that partitions the elements into groups.
partition
: Returns elements partitioned into two new Arrays, as determined by the given block.
slice_after
: Returns a new Enumerator whose entries are a partition of self
, based either on a given object
or a given block.
slice_before
: Returns a new Enumerator whose entries are a partition of self
, based either on a given object
or a given block.
slice_when
: Returns a new Enumerator whose entries are a partition of self
based on the given block.
chunk
: Returns elements organized into chunks as specified by the given block.
chunk_while
: Returns elements organized into chunks as specified by the given block.
These methods return elements that meet a specified criterion:
find_all
, filter
, select
: Returns elements selected by the block.
find_index
: Returns the index of an element selected by a given object or block.
reject
: Returns elements not rejected by the block.
uniq
: Returns elements that are not duplicates.
These methods return elements in sorted order:
sort
: Returns the elements, sorted by <=>
or the given block.
sort_by
: Returns the elements, sorted by the given block.
each_entry
: Calls the block with each successive element (slightly different from each).
each_with_index
: Calls the block with each successive element and its index.
each_with_object
: Calls the block with each successive element and a given object.
each_slice
: Calls the block with successive non-overlapping slices.
each_cons
: Calls the block with successive overlapping slices. (different from each_slice
).
reverse_each
: Calls the block with each successive element, in reverse order.
filter_map
: Returns truthy objects returned by the block.
flat_map
, collect_concat
: Returns flattened objects returned by the block.
grep
: Returns elements selected by a given object or objects returned by a given block.
grep_v
: Returns elements selected by a given object or objects returned by a given block.
reduce
, inject
: Returns the object formed by combining all elements.
sum
: Returns the sum of the elements, using method +
.
zip
: Combines each element with elements from other enumerables; returns the n-tuples or calls the block with each.
cycle
: Calls the block with each element, cycling repeatedly.
To use module Enumerable in a collection class:
Include it:
include Enumerable
Implement method each which must yield successive elements of the collection. The method will be called by almost any Enumerable method.
Example:
class Foo include Enumerable def each yield 1 yield 1, 2 yield end end Foo.new.each_entry{ |element| p element }
Output:
1 [1, 2] nil
These Ruby core classes include (or extend) Enumerable:
These Ruby standard library classes include Enumerable:
Virtually all methods in Enumerable call method each in the including class:
Hash#each
yields the next key-value pair as a 2-element Array.
Struct#each
yields the next name-value pair as a 2-element Array.
For the other classes above, each yields the next object from the collection.
The example code snippets for the Enumerable methods:
Always show the use of one or more Array-like classes (often Array itself).
Sometimes show the use of a Hash-like class. For some methods, though, the usage would not make sense, and so it is not shown. Example: tally
would find exactly one of each Hash entry.
Ruby exception objects are subclasses of Exception
. However, operating systems typically report errors using plain integers. Module
Errno
is created dynamically to map these operating system errors to Ruby classes, with each error number generating its own subclass of SystemCallError
. As the subclass is created in module Errno
, its name will start Errno::
.
The names of the Errno::
classes depend on the environment in which Ruby runs. On a typical Unix or Windows platform, there are Errno
classes such as Errno::EACCES, Errno::EAGAIN, Errno::EINTR, and so on.
The integer operating system error number corresponding to a particular error is available as the class constant Errno::
error::Errno
.
Errno::EACCES::Errno #=> 13 Errno::EAGAIN::Errno #=> 11 Errno::EINTR::Errno #=> 4
The full list of operating system errors on your particular platform are available as the constants of Errno
.
Errno.constants #=> :E2BIG, :EACCES, :EADDRINUSE, :EADDRNOTAVAIL, ...
The Warning
module contains a single method named warn
, and the module extends itself, making Warning.warn
available. Warning.warn
is called for all warnings issued by Ruby. By default, warnings are printed to $stderr.
Changing the behavior of Warning.warn
is useful to customize how warnings are handled by Ruby, for instance by filtering some warnings, and/or outputting warnings somewhere other than $stderr.
If you want to change the behavior of Warning.warn
you should use +Warning.extend(MyNewModuleWithWarnMethod)+ and you can use ‘super` to get the default behavior of printing the warning to $stderr.
Example:
module MyWarningFilter def warn(message, category: nil, **kwargs) if /some warning I want to ignore/.match?(message) # ignore else super end end end Warning.extend MyWarningFilter
You should never redefine Warning#warn
(the instance method), as that will then no longer provide a way to use the default behavior.
The warning
gem provides convenient ways to customize Warning.warn
.
Coverage
provides coverage measurement feature for Ruby. This feature is experimental, so these APIs may be changed in future.
Caveat: Currently, only process-global coverage measurement is supported. You cannot measure per-thread coverage.
require “coverage”
require or load Ruby source file
Coverage.result
will return a hash that contains filename as key and coverage array as value. A coverage array gives, for each line, the number of line execution by the interpreter. A nil
value means coverage is disabled for this line (lines like else
and end
).
[foo.rb] s = 0 10.times do |x| s += x end if s == 45 p :ok else p :ng end [EOF] require "coverage" Coverage.start require "foo.rb" p Coverage.result #=> {"foo.rb"=>[1, 1, 10, nil, nil, 1, 1, nil, 0, nil]}
Coverage
If a coverage mode is not explicitly specified when starting coverage, lines coverage is what will run. It reports the number of line executions for each line.
require "coverage" Coverage.start(lines: true) require "foo.rb" p Coverage.result #=> {"foo.rb"=>{:lines=>[1, 1, 10, nil, nil, 1, 1, nil, 0, nil]}}
The value of the lines coverage result is an array containing how many times each line was executed. Order in this array is important. For example, the first item in this array, at index 0, reports how many times line 1 of this file was executed while coverage was run (which, in this example, is one time).
A nil
value means coverage is disabled for this line (lines like else
and end
).
Coverage
Oneshot lines coverage tracks and reports on the executed lines while coverage is running. It will not report how many times a line was executed, only that it was executed.
require "coverage" Coverage.start(oneshot_lines: true) require "foo.rb" p Coverage.result #=> {"foo.rb"=>{:oneshot_lines=>[1, 2, 3, 6, 7]}}
The value of the oneshot lines coverage result is an array containing the line numbers that were executed.
Coverage
Branches coverage reports how many times each branch within each conditional was executed.
require "coverage" Coverage.start(branches: true) require "foo.rb" p Coverage.result #=> {"foo.rb"=>{:branches=>{[:if, 0, 6, 0, 10, 3]=>{[:then, 1, 7, 2, 7, 7]=>1, [:else, 2, 9, 2, 9, 7]=>0}}}}
Each entry within the branches hash is a conditional, the value of which is another hash where each entry is a branch in that conditional. The values are the number of times the method was executed, and the keys are identifying information about the branch.
The information that makes up each key identifying branches or conditionals is the following, from left to right:
A label for the type of branch or conditional.
A unique identifier.
The starting line number it appears on in the file.
The starting column number it appears on in the file.
The ending line number it appears on in the file.
The ending column number it appears on in the file.
Coverage
Methods coverage reports how many times each method was executed.
[foo_method.rb] class Greeter def greet "welcome!" end end def hello "Hi" end hello() Greeter.new.greet() [EOF] require "coverage" Coverage.start(methods: true) require "foo_method.rb" p Coverage.result #=> {"foo_method.rb"=>{:methods=>{[Object, :hello, 7, 0, 9, 3]=>1, [Greeter, :greet, 2, 2, 4, 5]=>1}}}
Each entry within the methods hash represents a method. The values in this hash are the number of times the method was executed, and the keys are identifying information about the method.
The information that makes up each key identifying a method is the following, from left to right:
The class.
The method name.
The starting line number the method appears on in the file.
The starting column number the method appears on in the file.
The ending line number the method appears on in the file.
The ending column number the method appears on in the file.
Coverage
Modes You can also run all modes of coverage simultaneously with this shortcut. Note that running all coverage modes does not run both lines and oneshot lines. Those modes cannot be run simultaneously. Lines coverage is run in this case, because you can still use it to determine whether or not a line was executed.
require "coverage" Coverage.start(:all) require "foo.rb" p Coverage.result #=> {"foo.rb"=>{:lines=>[1, 1, 10, nil, nil, 1, 1, nil, 0, nil], :branches=>{[:if, 0, 6, 0, 10, 3]=>{[:then, 1, 7, 2, 7, 7]=>1, [:else, 2, 9, 2, 9, 7]=>0}}, :methods=>{}}}
Racc
is a LALR(1) parser generator. It is written in Ruby itself, and generates Ruby programs.
racc [-o<var>filename</var>] [--output-file=<var>filename</var>] [-e<var>rubypath</var>] [--executable=<var>rubypath</var>] [-v] [--verbose] [-O<var>filename</var>] [--log-file=<var>filename</var>] [-g] [--debug] [-E] [--embedded] [-l] [--no-line-convert] [-c] [--line-convert-all] [-a] [--no-omit-actions] [-C] [--check-only] [-S] [--output-status] [--version] [--copyright] [--help] <var>grammarfile</var>
grammarfile
Racc
grammar file. Any extension is permitted.
outfile
A filename for output. default is <filename
>.tab.rb
filename
Place logging output in file filename
. Default log file name is <filename
>.output.
rubypath
output executable file(mode 755). where path
is the Ruby interpreter.
verbose mode. create filename
.output file, like yacc’s y.output file.
add debug code to parser class. To display debuggin information, use this ‘-g’ option and set @yydebug true in parser class.
Output parser which doesn’t need runtime files (racc/parser.rb).
Check syntax of racc grammar file and quit.
Print messages time to time while compiling.
turns off line number converting.
Convert line number of actions, inner, header and footer.
Call all actions, even if an action is empty.
print Racc
version and quit.
Print copyright and quit.
Print usage and quit.
Parser
Using Racc
To compile Racc
grammar file, simply type:
$ racc parse.y
This creates Ruby script file “parse.tab.y”. The -o option can change the output filename.
Racc
Grammar
File
If you want your own parser, you have to write a grammar file. A grammar file contains the name of your parser class, grammar for the parser, user code, and anything else. When writing a grammar file, yacc’s knowledge is helpful. If you have not used yacc before, Racc
is not too difficult.
Here’s an example Racc
grammar file.
class Calcparser rule target: exp { print val[0] } exp: exp '+' exp | exp '*' exp | '(' exp ')' | NUMBER end
Racc
grammar files resemble yacc files. But (of course), this is Ruby code. yacc’s $$ is the ‘result’, $0, $1… is an array called ‘val’, and $-1, $-2… is an array called ‘_values’.
See the Grammar File Reference for more information on grammar files.
Parser
Then you must prepare the parse entry method. There are two types of parse methods in Racc
, Racc::Parser#do_parse and Racc::Parser#yyparse
Racc::Parser#do_parse is simple.
It’s yyparse() of yacc, and Racc::Parser#next_token
is yylex(). This method must returns an array like [TOKENSYMBOL, ITS_VALUE]. EOF is [false, false]. (TOKENSYMBOL is a Ruby symbol (taken from String#intern
) by default. If you want to change this, see the grammar reference.
Racc::Parser#yyparse is little complicated, but useful. It does not use Racc::Parser#next_token
, instead it gets tokens from any iterator.
For example, yyparse(obj, :scan)
causes calling +obj#scan+, and you can return tokens by yielding them from +obj#scan+.
When debugging, “-v” or/and the “-g” option is helpful.
“-v” creates verbose log file (.output). “-g” creates a “Verbose Parser”. Verbose Parser
prints the internal status when parsing. But it’s not automatic. You must use -g option and set +@yydebug+ to true
in order to get output. -g option only creates the verbose parser.
Racc
reported syntax error. Isn’t there too many “end”? grammar of racc file is changed in v0.10.
Racc
does not use ‘%’ mark, while yacc uses huge number of ‘%’ marks..
Racc
reported “XXXX conflicts”. Try “racc -v xxxx.y”. It causes producing racc’s internal log file, xxxx.output.
Try “racc -g xxxx.y”. This command let racc generate “debugging parser”. Then set @yydebug=true in your parser. It produces a working log of your parser.
Racc
runtime A parser, which is created by Racc
, requires the Racc
runtime module; racc/parser.rb.
Ruby 1.8.x comes with Racc
runtime module, you need NOT distribute Racc
runtime files.
If you want to include the Racc
runtime module with your parser. This can be done by using ‘-E’ option:
$ racc -E -omyparser.rb myparser.y
This command creates myparser.rb which ‘includes’ Racc
runtime. Only you must do is to distribute your parser file (myparser.rb).
Note: parser.rb is ruby license, but your parser is not. Your own parser is completely yours.
The Readline
module provides interface for GNU Readline
. This module defines a number of methods to facilitate completion and accesses input history from the Ruby interpreter. This module supported Edit Line(libedit) too. libedit is compatible with GNU Readline
.
Reads one inputted line with line edit by Readline.readline
method. At this time, the facilitatation completion and the key bind like Emacs can be operated like GNU Readline
.
require "readline" while buf = Readline.readline("> ", true) p buf end
The content that the user input can be recorded to the history. The history can be accessed by Readline::HISTORY
constant.
require "readline" while buf = Readline.readline("> ", true) p Readline::HISTORY.to_a print("-> ", buf, "\n") end
Documented by Kouji Takao <kouji dot takao at gmail dot com>.
The Benchmark
module provides methods to measure and report the time used to execute Ruby code.
Measure the time to construct the string given by the expression "a"*1_000_000_000
:
require 'benchmark' puts Benchmark.measure { "a"*1_000_000_000 }
On my machine (OSX 10.8.3 on i5 1.7 GHz) this generates:
0.350000 0.400000 0.750000 ( 0.835234)
This report shows the user CPU time, system CPU time, the sum of the user and system CPU times, and the elapsed real time. The unit of time is seconds.
Do some experiments sequentially using the bm
method:
require 'benchmark' n = 5000000 Benchmark.bm do |x| x.report { for i in 1..n; a = "1"; end } x.report { n.times do ; a = "1"; end } x.report { 1.upto(n) do ; a = "1"; end } end
The result:
user system total real 1.010000 0.000000 1.010000 ( 1.014479) 1.000000 0.000000 1.000000 ( 0.998261) 0.980000 0.000000 0.980000 ( 0.981335)
Continuing the previous example, put a label in each report:
require 'benchmark' n = 5000000 Benchmark.bm(7) do |x| x.report("for:") { for i in 1..n; a = "1"; end } x.report("times:") { n.times do ; a = "1"; end } x.report("upto:") { 1.upto(n) do ; a = "1"; end } end
The result:
user system total real for: 1.010000 0.000000 1.010000 ( 1.015688) times: 1.000000 0.000000 1.000000 ( 1.003611) upto: 1.030000 0.000000 1.030000 ( 1.028098)
The times for some benchmarks depend on the order in which items are run. These differences are due to the cost of memory allocation and garbage collection. To avoid these discrepancies, the bmbm
method is provided. For example, to compare ways to sort an array of floats:
require 'benchmark' array = (1..1000000).map { rand } Benchmark.bmbm do |x| x.report("sort!") { array.dup.sort! } x.report("sort") { array.dup.sort } end
The result:
Rehearsal ----------------------------------------- sort! 1.490000 0.010000 1.500000 ( 1.490520) sort 1.460000 0.000000 1.460000 ( 1.463025) -------------------------------- total: 2.960000sec user system total real sort! 1.460000 0.000000 1.460000 ( 1.460465) sort 1.450000 0.010000 1.460000 ( 1.448327)
Report statistics of sequential experiments with unique labels, using the benchmark
method:
require 'benchmark' include Benchmark # we need the CAPTION and FORMAT constants n = 5000000 Benchmark.benchmark(CAPTION, 7, FORMAT, ">total:", ">avg:") do |x| tf = x.report("for:") { for i in 1..n; a = "1"; end } tt = x.report("times:") { n.times do ; a = "1"; end } tu = x.report("upto:") { 1.upto(n) do ; a = "1"; end } [tf+tt+tu, (tf+tt+tu)/3] end
The result:
user system total real for: 0.950000 0.000000 0.950000 ( 0.952039) times: 0.980000 0.000000 0.980000 ( 0.984938) upto: 0.950000 0.000000 0.950000 ( 0.946787) >total: 2.880000 0.000000 2.880000 ( 2.883764) >avg: 0.960000 0.000000 0.960000 ( 0.961255)
The Forwardable
module provides delegation of specified methods to a designated object, using the methods def_delegator
and def_delegators
.
For example, say you have a class RecordCollection which contains an array @records
. You could provide the lookup method record_number(), which simply calls [] on the @records
array, like this:
require 'forwardable' class RecordCollection attr_accessor :records extend Forwardable def_delegator :@records, :[], :record_number end
We can use the lookup method like so:
r = RecordCollection.new r.records = [4,5,6] r.record_number(0) # => 4
Further, if you wish to provide the methods size, <<, and map, all of which delegate to @records, this is how you can do it:
class RecordCollection # re-open RecordCollection class def_delegators :@records, :size, :<<, :map end r = RecordCollection.new r.records = [1,2,3] r.record_number(0) # => 1 r.size # => 3 r << 4 # => [1, 2, 3, 4] r.map { |x| x * 2 } # => [2, 4, 6, 8]
You can even extend regular objects with Forwardable
.
my_hash = Hash.new my_hash.extend Forwardable # prepare object for delegation my_hash.def_delegator "STDOUT", "puts" # add delegation for STDOUT.puts() my_hash.puts "Howdy!"
You could use Forwardable
as an alternative to inheritance, when you don’t want to inherit all methods from the superclass. For instance, here is how you might add a range of Array
instance methods to a new class Queue
:
class Queue extend Forwardable def initialize @q = [ ] # prepare delegate object end # setup preferred interface, enq() and deq()... def_delegator :@q, :push, :enq def_delegator :@q, :shift, :deq # support some general Array methods that fit Queues well def_delegators :@q, :clear, :first, :push, :shift, :size end q = Thread::Queue.new q.enq 1, 2, 3, 4, 5 q.push 6 q.shift # => 1 while q.size > 0 puts q.deq end q.enq "Ruby", "Perl", "Python" puts q.first q.clear puts q.first
This should output:
2 3 4 5 6 Ruby nil
Be advised, RDoc
will not detect delegated methods.
forwardable.rb
provides single-method delegation via the def_delegator
and def_delegators
methods. For full-class delegation via DelegateClass, see delegate.rb
.
SingleForwardable
can be used to setup delegation at the object level as well.
printer = String.new printer.extend SingleForwardable # prepare object for delegation printer.def_delegator "STDOUT", "puts" # add delegation for STDOUT.puts() printer.puts "Howdy!"
Also, SingleForwardable
can be used to set up delegation for a Class
or Module
.
class Implementation def self.service puts "serviced!" end end module Facade extend SingleForwardable def_delegator :Implementation, :service end Facade.service #=> serviced!
If you want to use both Forwardable
and SingleForwardable
, you can use methods def_instance_delegator and def_single_delegator
, etc.
This library is an interface to secure random number generators which are suitable for generating session keys in HTTP cookies, etc.
You can use this library in your application by requiring it:
require 'securerandom'
It supports the following secure random number generators:
openssl
/dev/urandom
SecureRandom
is extended by the Random::Formatter
module which defines the following methods:
alphanumeric
base64
choose
hex
rand
random_bytes
random_number
urlsafe_base64
uuid
These methods are usable as class methods of SecureRandom
such as SecureRandom.hex
.
If a secure random number generator is not available, NotImplementedError
is raised.
The marshaling library converts collections of Ruby objects into a byte stream, allowing them to be stored outside the currently active script. This data may subsequently be read and the original objects reconstituted.
Marshaled data has major and minor version numbers stored along with the object information. In normal use, marshaling can only load data written with the same major version number and an equal or lower minor version number. If Ruby’s “verbose” flag is set (normally using -d, -v, -w, or –verbose) the major and minor numbers must match exactly. Marshal
versioning is independent of Ruby’s version numbers. You can extract the version by reading the first two bytes of marshaled data.
str = Marshal.dump("thing") RUBY_VERSION #=> "1.9.0" str[0].ord #=> 4 str[1].ord #=> 8
Some objects cannot be dumped: if the objects to be dumped include bindings, procedure or method objects, instances of class IO
, or singleton objects, a TypeError
will be raised.
If your class has special serialization needs (for example, if you want to serialize in some specific format), or if it contains objects that would otherwise not be serializable, you can implement your own serialization strategy.
There are two methods of doing this, your object can define either marshal_dump and marshal_load or _dump and _load. marshal_dump will take precedence over _dump if both are defined. marshal_dump may result in smaller Marshal
strings.
By design, Marshal.load
can deserialize almost any class loaded into the Ruby process. In many cases this can lead to remote code execution if the Marshal
data is loaded from an untrusted source.
As a result, Marshal.load
is not suitable as a general purpose serialization format and you should never unmarshal user supplied input or other untrusted data.
If you need to deserialize untrusted data, use JSON
or another serialization format that is only able to load simple, ‘primitive’ types such as String
, Array
, Hash
, etc. Never allow user input to specify arbitrary types to deserialize into.
When dumping an object the method marshal_dump will be called. marshal_dump must return a result containing the information necessary for marshal_load to reconstitute the object. The result can be any object.
When loading an object dumped using marshal_dump the object is first allocated then marshal_load is called with the result from marshal_dump. marshal_load must recreate the object from the information in the result.
Example:
class MyObj def initialize name, version, data @name = name @version = version @data = data end def marshal_dump [@name, @version] end def marshal_load array @name, @version = array end end
Use _dump and _load when you need to allocate the object you’re restoring yourself.
When dumping an object the instance method _dump is called with an Integer
which indicates the maximum depth of objects to dump (a value of -1 implies that you should disable depth checking). _dump must return a String
containing the information necessary to reconstitute the object.
The class method _load should take a String
and use it to return an object of the same class.
Example:
class MyObj def initialize name, version, data @name = name @version = version @data = data end def _dump level [@name, @version].join ':' end def self._load args new(*args.split(':')) end end
Since Marshal.dump
outputs a string you can have _dump return a Marshal
string which is Marshal.loaded in _load for complex objects.