Ripper is a Ruby script parser.
You can get information from the parser with event-based style. Information such as abstract syntax trees or simple lexical analysis of the Ruby program.
Ripper provides an easy interface for parsing your program into a symbolic expression tree (or S-expression).
Understanding the output of the parser may come as a challenge, it’s recommended you use PP to format the output for legibility.
require 'ripper'
require 'pp'
pp Ripper.sexp('def hello(world) "Hello, #{world}!"; end')
#=> [:program,
[[:def,
[:@ident, "hello", [1, 4]],
[:paren,
[:params, [[:@ident, "world", [1, 10]]], nil, nil, nil, nil, nil, nil]],
[:bodystmt,
[[:string_literal,
[:string_content,
[:@tstring_content, "Hello, ", [1, 18]],
[:string_embexpr, [[:var_ref, [:@ident, "world", [1, 27]]]]],
[:@tstring_content, "!", [1, 33]]]]],
nil,
nil,
nil]]]]
You can see in the example above, the expression starts with :program.
From here, a method definition at :def, followed by the method’s identifier :@ident. After the method’s identifier comes the parentheses :paren and the method parameters under :params.
Next is the method body, starting at :bodystmt (stmt meaning statement), which contains the full definition of the method.
In our case, we’re simply returning a String, so next we have the :string_literal expression.
Within our :string_literal you’ll notice two @tstring_content, this is the literal part for Hello, and !. Between the two @tstring_content statements is a :string_embexpr, where embexpr is an embedded expression. Our expression consists of a local variable, or var_ref, with the identifier (@ident) of world.
ruby 1.9 (support CVS HEAD only)
bison 1.28 or later (Other yaccs do not work)
Ruby License. Minero Aoki aamine@loveruby.net http://i.loveruby.net
SocketError is the error class for socket.
Pseudo I/O on String object.
Commonly used to simulate ‘$stdio` or `$stderr`
require 'stringio' io = StringIO.new io.puts "Hello World" io.string #=> "Hello World\n"
StringScanner provides for lexical scanning operations on a String. Here is an example of its usage:
s = StringScanner.new('This is an example string') s.eos? # -> false p s.scan(/\w+/) # -> "This" p s.scan(/\w+/) # -> nil p s.scan(/\s+/) # -> " " p s.scan(/\s+/) # -> nil p s.scan(/\w+/) # -> "is" s.eos? # -> false p s.scan(/\s+/) # -> " " p s.scan(/\w+/) # -> "an" p s.scan(/\s+/) # -> " " p s.scan(/\w+/) # -> "example" p s.scan(/\s+/) # -> " " p s.scan(/\w+/) # -> "string" s.eos? # -> true p s.scan(/\s+/) # -> nil p s.scan(/\w+/) # -> nil
Scanning a string means remembering the position of a scan pointer, which is just an index. The point of scanning is to move forward a bit at a time, so matches are sought after the scan pointer; usually immediately after it.
Given the string “test string”, here are the pertinent scan pointer positions:
t e s t s t r i n g
0 1 2 ... 1
0
When you scan for a pattern (a regular expression), the match must occur at the character after the scan pointer. If you use scan_until, then the match can occur anywhere after the scan pointer. In both cases, the scan pointer moves just beyond the last character of the match, ready to scan again from the next character onwards. This is demonstrated by the example above.
Method Categories There are other methods besides the plain scanners. You can look ahead in the string without actually scanning. You can access the most recent match. You can modify the string being scanned, reset or terminate the scanner, find out or change the position of the scan pointer, skip ahead, and so on.
beginning_of_line? (bol?)
Data There are aliases to several of the methods.
Raised by some IO operations when reaching the end of file. Many IO methods exist in two forms,
one that returns nil when the end of file is reached, the other raises EOFError.
EOFError is a subclass of IOError.
file = File.open("/etc/hosts") file.read file.gets #=> nil file.readline #=> EOFError: end of file reached
ARGF is a stream designed for use in scripts that process files given as command-line arguments or passed in via STDIN.
The arguments passed to your script are stored in the ARGV Array, one argument per element. ARGF assumes that any arguments that aren’t filenames have been removed from ARGV. For example:
$ ruby argf.rb --verbose file1 file2 ARGV #=> ["--verbose", "file1", "file2"] option = ARGV.shift #=> "--verbose" ARGV #=> ["file1", "file2"]
You can now use ARGF to work with a concatenation of each of these named files. For instance, ARGF.read will return the contents of file1 followed by the contents of file2.
After a file in ARGV has been read ARGF removes it from the Array. Thus, after all files have been read ARGV will be empty.
You can manipulate ARGV yourself to control what ARGF operates on. If you remove a file from ARGV, it is ignored by ARGF; if you add files to ARGV, they are treated as if they were named on the command line. For example:
ARGV.replace ["file1"] ARGF.readlines # Returns the contents of file1 as an Array ARGV #=> [] ARGV.replace ["file2", "file3"] ARGF.read # Returns the contents of file2 and file3
If ARGV is empty, ARGF acts as if it contained STDIN, i.e. the data piped to your script. For example:
$ echo "glark" | ruby -e 'p ARGF.read' "glark\n"
Outputs a source level execution trace of a Ruby program.
It does this by registering an event handler with Kernel#set_trace_func for processing incoming events. It also provides methods for filtering unwanted trace output (see Tracer.add_filter, Tracer.on, and Tracer.off).
Consider the following Ruby script
class A def square(a) return a*a end end a = A.new a.square(5)
Running the above script using ruby -r tracer example.rb will output the following trace to STDOUT (Note you can also explicitly require 'tracer')
#0:<internal:lib/rubygems/custom_require>:38:Kernel:<: - #0:example.rb:3::-: class A #0:example.rb:3::C: class A #0:example.rb:4::-: def square(a) #0:example.rb:7::E: end #0:example.rb:9::-: a = A.new #0:example.rb:10::-: a.square(5) #0:example.rb:4:A:>: def square(a) #0:example.rb:5:A:-: return a*a #0:example.rb:6:A:<: end | | | | | | | | | ---------------------+ event | | | ------------------------+ class | | --------------------------+ line | ------------------------------------+ filename ---------------------------------------+ thread
Symbol table used for displaying incoming events:
call a C-language routine
return from a C-language routine
call a Ruby method
C
start a class or module definition
E
finish a class or module definition
-
execute code on a new line
raise an exception
return from a Ruby method
by Keiju ISHITSUKA(keiju@ishitsuka.com)
This library provides debugging functionality to Ruby.
To add a debugger to your code, start by requiring debug in your program:
def say(word) require 'debug' puts word end
This will cause Ruby to interrupt execution and show a prompt when the say method is run.
Once you’re inside the prompt, you can start debugging your program.
(rdb:1) p word "hello"
You can get help at any time by pressing h.
(rdb:1) h
Debugger help v.-0.002b
Commands
b[reak] [file:|class:]<line|method>
b[reak] [class.]<line|method>
set breakpoint to some position
wat[ch] <expression> set watchpoint to some expression
cat[ch] (<exception>|off) set catchpoint to an exception
b[reak] list breakpoints
cat[ch] show catchpoint
del[ete][ nnn] delete some or all breakpoints
disp[lay] <expression> add expression into display expression list
undisp[lay][ nnn] delete one particular or all display expressions
c[ont] run until program ends or hit breakpoint
s[tep][ nnn] step (into methods) one line or till line nnn
n[ext][ nnn] go over one line or till line nnn
w[here] display frames
f[rame] alias for where
l[ist][ (-|nn-mm)] list program, - lists backwards
nn-mm lists given lines
up[ nn] move to higher frame
down[ nn] move to lower frame
fin[ish] return to outer frame
tr[ace] (on|off) set trace mode of current thread
tr[ace] (on|off) all set trace mode of all threads
q[uit] exit from debugger
v[ar] g[lobal] show global variables
v[ar] l[ocal] show local variables
v[ar] i[nstance] <object> show instance variables of object
v[ar] c[onst] <object> show constants of object
m[ethod] i[nstance] <obj> show methods of object
m[ethod] <class|module> show instance methods of class or module
th[read] l[ist] list all threads
th[read] c[ur[rent]] show current thread
th[read] [sw[itch]] <nnn> switch thread context to nnn
th[read] stop <nnn> stop thread nnn
th[read] resume <nnn> resume thread nnn
p expression evaluate expression and print its value
h[elp] print this help
<everything else> evaluate
The following is a list of common functionalities that the debugger provides.
In general, a debugger is used to find bugs in your program, which often means pausing execution and inspecting variables at some point in time.
Let’s look at an example:
def my_method(foo) require 'debug' foo = get_foo if foo.nil? raise if foo.nil? end
When you run this program, the debugger will kick in just before the foo assignment.
(rdb:1) p foo nil
In this example, it’d be interesting to move to the next line and inspect the value of foo again. You can do that by pressing n:
(rdb:1) n # goes to next line (rdb:1) p foo nil
You now know that the original value of foo was nil, and that it still was nil after calling get_foo.
Other useful commands for navigating through your code are:
c
Runs the program until it either exists or encounters another breakpoint. You usually press c when you are finished debugging your program and want to resume its execution.
s
Steps into method definition. In the previous example, s would take you inside the method definition of get_foo.
r
Restart the program.
q
Quit the program.
You can use the debugger to easily inspect both local and global variables. We’ve seen how to inspect local variables before:
(rdb:1) p my_arg 42
You can also pretty print the result of variables or expressions:
(rdb:1) pp %w{a very long long array containing many words}
["a",
"very",
"long",
...
]
You can list all local variables with +v l+:
(rdb:1) v l foo => "hello"
Similarly, you can show all global variables with +v g+:
(rdb:1) v g all global variables
Finally, you can omit p if you simply want to evaluate a variable or expression
(rdb:1) 5**2 25
Ruby Debug provides more advanced functionalities like switching between threads, setting breakpoints and watch expressions, and more. The full list of commands is available at any time by pressing h.
Make sure you remove every instance of +require ‘debug’+ before shipping your code. Failing to do so may result in your program hanging unpredictably.
Debug is not available in safe mode.
ERB – Ruby Templating ERB provides an easy to use but powerful templating system for Ruby. Using ERB, actual Ruby code can be added to any plain text document for the purposes of generating document information details and/or flow control.
A very simple example is this:
require 'erb' x = 42 template = ERB.new <<-EOF The value of x is: <%= x %> EOF puts template.result(binding)
Prints: The value of x is: 42
More complex examples are given below.
ERB recognizes certain tags in the provided template and converts them based on the rules below:
<% Ruby code -- inline with output %> <%= Ruby expression -- replace with result %> <%# comment -- ignored -- useful in testing %> % a line of Ruby code -- treated as <% line %> (optional -- see ERB.new) %% replaced with % if first thing on a line and % processing is used <%% or %%> -- replace with <% or %> respectively
All other text is passed through ERB filtering unchanged.
There are several settings you can change when you use ERB:
the nature of the tags that are recognized;
the value of $SAFE under which the template is run;
the binding used to resolve local variables in the template.
See the ERB.new and ERB#result methods for more detail.
ERB (or Ruby code generated by ERB) returns a string in the same character encoding as the input string. When the input string has a magic comment, however, it returns a string in the encoding specified by the magic comment.
# -*- coding: utf-8 -*- require 'erb' template = ERB.new <<EOF <%#-*- coding: Big5 -*-%> \_\_ENCODING\_\_ is <%= \_\_ENCODING\_\_ %>. EOF puts template.result
Prints: _ENCODING_ is Big5.
ERB is useful for any generic templating situation. Note that in this example, we use the convenient “% at start of line” tag, and we quote the template literally with %q{...} to avoid trouble with the backslash.
require "erb" # Create template. template = %q{ From: James Edward Gray II <james@grayproductions.net> To: <%= to %> Subject: Addressing Needs <%= to[/\w+/] %>: Just wanted to send a quick note assuring that your needs are being addressed. I want you to know that my team will keep working on the issues, especially: <%# ignore numerous minor requests -- focus on priorities %> % priorities.each do |priority| * <%= priority %> % end Thanks for your patience. James Edward Gray II }.gsub(/^ /, '') message = ERB.new(template, 0, "%<>") # Set up template data. to = "Community Spokesman <spokesman@ruby_community.org>" priorities = [ "Run Ruby Quiz", "Document Modules", "Answer Questions on Ruby Talk" ] # Produce result. email = message.result puts email
Generates:
From: James Edward Gray II <james@grayproductions.net>
To: Community Spokesman <spokesman@ruby_community.org>
Subject: Addressing Needs
Community:
Just wanted to send a quick note assuring that your needs are being addressed.
I want you to know that my team will keep working on the issues, especially:
* Run Ruby Quiz
* Document Modules
* Answer Questions on Ruby Talk
Thanks for your patience.
James Edward Gray II
ERB is often used in .rhtml files (HTML with embedded Ruby). Notice the need in this example to provide a special binding when the template is run, so that the instance variables in the Product object can be resolved.
require "erb" # Build template data class. class Product def initialize( code, name, desc, cost ) @code = code @name = name @desc = desc @cost = cost @features = [ ] end def add_feature( feature ) @features << feature end # Support templating of member data. def get_binding binding end # ... end # Create template. template = %{ <html> <head><title>Ruby Toys -- <%= @name %></title></head> <body> <h1><%= @name %> (<%= @code %>)</h1> <p><%= @desc %></p> <ul> <% @features.each do |f| %> <li><b><%= f %></b></li> <% end %> </ul> <p> <% if @cost < 10 %> <b>Only <%= @cost %>!!!</b> <% else %> Call for a price, today! <% end %> </p> </body> </html> }.gsub(/^ /, '') rhtml = ERB.new(template) # Set up template data. toy = Product.new( "TZ-1002", "Rubysapien", "Geek's Best Friend! Responds to Ruby commands...", 999.95 ) toy.add_feature("Listens for verbal commands in the Ruby language!") toy.add_feature("Ignores Perl, Java, and all C variants.") toy.add_feature("Karate-Chop Action!!!") toy.add_feature("Matz signature on left leg.") toy.add_feature("Gem studded eyes... Rubies, of course!") # Produce result. rhtml.run(toy.get_binding)
Generates (some blank lines removed):
<html>
<head><title>Ruby Toys -- Rubysapien</title></head>
<body>
<h1>Rubysapien (TZ-1002)</h1>
<p>Geek's Best Friend! Responds to Ruby commands...</p>
<ul>
<li><b>Listens for verbal commands in the Ruby language!</b></li>
<li><b>Ignores Perl, Java, and all C variants.</b></li>
<li><b>Karate-Chop Action!!!</b></li>
<li><b>Matz signature on left leg.</b></li>
<li><b>Gem studded eyes... Rubies, of course!</b></li>
</ul>
<p>
Call for a price, today!
</p>
</body>
</html>
There are a variety of templating solutions available in various Ruby projects:
ERB’s big brother, eRuby, works the same but is written in C for speed;
Amrita (smart at producing HTML/XML);
cs/Template (written in C for speed);
RDoc, distributed with Ruby, uses its own template engine, which can be reused elsewhere;
and others; search RubyGems.org or The Ruby Toolbox.
Rails, the web application framework, uses ERB to create views.
IPAddr provides a set of methods to manipulate an IP address. Both IPv4 and IPv6 are supported.
require 'ipaddr' ipaddr1 = IPAddr.new "3ffe:505:2::1" p ipaddr1 #=> #<IPAddr: IPv6:3ffe:0505:0002:0000:0000:0000:0000:0001/ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff> p ipaddr1.to_s #=> "3ffe:505:2::1" ipaddr2 = ipaddr1.mask(48) #=> #<IPAddr: IPv6:3ffe:0505:0002:0000:0000:0000:0000:0000/ffff:ffff:ffff:0000:0000:0000:0000:0000> p ipaddr2.to_s #=> "3ffe:505:2::" ipaddr3 = IPAddr.new "192.168.2.0/24" p ipaddr3 #=> #<IPAddr: IPv4:192.168.2.0/255.255.255.0>
The Logger class provides a simple but sophisticated logging utility that you can use to output messages.
The messages have associated levels, such as INFO or ERROR that indicate their importance. You can then give the Logger a level, and only messages at that level or higher will be printed.
The levels are:
UNKNOWN
An unknown message that should always be logged.
FATAL
An unhandleable error that results in a program crash.
ERROR
A handleable error condition.
WARN
A warning.
INFO
Generic (useful) information about system operation.
DEBUG
Low-level information for developers.
For instance, in a production system, you may have your Logger set to INFO or even WARN. When you are developing the system, however, you probably want to know about the program’s internal state, and would set the Logger to DEBUG.
Note: Logger does not escape or sanitize any messages passed to it. Developers should be aware of when potentially malicious data (user-input) is passed to Logger, and manually escape the untrusted data:
logger.info("User-input: #{input.dump}") logger.info("User-input: %p" % input)
You can use formatter= for escaping all data.
original_formatter = Logger::Formatter.new logger.formatter = proc { |severity, datetime, progname, msg| original_formatter.call(severity, datetime, progname, msg.dump) } logger.info(input)
This creates a Logger that outputs to the standard output stream, with a level of WARN:
require 'logger' logger = Logger.new(STDOUT) logger.level = Logger::WARN logger.debug("Created logger") logger.info("Program started") logger.warn("Nothing to do!") path = "a_non_existent_file" begin File.foreach(path) do |line| unless line =~ /^(\w+) = (.*)$/ logger.error("Line in wrong format: #{line.chomp}") end end rescue => err logger.fatal("Caught exception; exiting") logger.fatal(err) end
Because the Logger’s level is set to WARN, only the warning, error, and fatal messages are recorded. The debug and info messages are silently discarded.
There are several interesting features that Logger provides, like auto-rolling of log files, setting the format of log messages, and specifying a program name in conjunction with the message. The next section shows you how to achieve these things.
The options below give you various choices, in more or less increasing complexity.
Create a logger which logs messages to STDERR/STDOUT.
logger = Logger.new(STDERR) logger = Logger.new(STDOUT)
Create a logger for the file which has the specified name.
logger = Logger.new('logfile.log')
Create a logger for the specified file.
file = File.open('foo.log', File::WRONLY | File::APPEND) # To create new (and to remove old) logfile, add File::CREAT like: # file = File.open('foo.log', File::WRONLY | File::APPEND | File::CREAT) logger = Logger.new(file)
Create a logger which ages the logfile once it reaches a certain size. Leave 10 “old” log files where each file is about 1,024,000 bytes.
logger = Logger.new('foo.log', 10, 1024000)
Create a logger which ages the logfile daily/weekly/monthly.
logger = Logger.new('foo.log', 'daily') logger = Logger.new('foo.log', 'weekly') logger = Logger.new('foo.log', 'monthly')
Notice the different methods (fatal, error, info) being used to log messages of various levels? Other methods in this family are warn and debug. add is used below to log a message of an arbitrary (perhaps dynamic) level.
Message in a block.
logger.fatal { "Argument 'foo' not given." }
Message as a string.
logger.error "Argument #{@foo} mismatch."
With progname.
logger.info('initialize') { "Initializing..." }
With severity.
logger.add(Logger::FATAL) { 'Fatal error!' }
The block form allows you to create potentially complex log messages, but to delay their evaluation until and unless the message is logged. For example, if we have the following:
logger.debug { "This is a " + potentially + " expensive operation" }
If the logger’s level is INFO or higher, no debug messages will be logged, and the entire block will not even be evaluated. Compare to this:
logger.debug("This is a " + potentially + " expensive operation")
Here, the string concatenation is done every time, even if the log level is not set to show the debug message.
logger.close
Original interface.
logger.sev_threshold = Logger::WARN
Log4r (somewhat) compatible interface.
logger.level = Logger::INFO # DEBUG < INFO < WARN < ERROR < FATAL < UNKNOWN
Symbol or String (case insensitive)
logger.level = :info logger.level = 'INFO' # :debug < :info < :warn < :error < :fatal < :unknown
Constructor
Logger.new(logdev, level: Logger::INFO) Logger.new(logdev, level: :info) Logger.new(logdev, level: 'INFO')
Log messages are rendered in the output stream in a certain format by default. The default format and a sample are shown below:
Log format:
SeverityID, [DateTime #pid] SeverityLabel -- ProgName: message
Log sample:
I, [1999-03-03T02:34:24.895701 #19074] INFO -- Main: info.
You may change the date and time format via datetime_format=.
logger.datetime_format = '%Y-%m-%d %H:%M:%S' # e.g. "2004-01-03 00:54:26"
or via the constructor.
Logger.new(logdev, datetime_format: '%Y-%m-%d %H:%M:%S')
Or, you may change the overall format via the formatter= method.
logger.formatter = proc do |severity, datetime, progname, msg| "#{datetime}: #{msg}\n" end # e.g. "2005-09-22 08:51:08 +0900: hello world"
or via the constructor.
Logger.new(logdev, formatter: proc {|severity, datetime, progname, msg| "#{datetime}: #{msg}\n" })
Use the Monitor class when you want to have a lock object for blocks with mutual exclusion.
require 'monitor' lock = Monitor.new lock.synchronize do # exclusive access end
A StringIO duck-typed class that uses Tempfile instead of String as the backing store.
This is available when rubygems/test_utilities is required.
Set implements a collection of unordered values with no duplicates. This is a hybrid of Array’s intuitive inter-operation facilities and Hash’s fast lookup.
Set is easy to use with Enumerable objects (implementing each). Most of the initializer methods and binary operators accept generic Enumerable objects besides sets and arrays. An Enumerable object can be converted to Set using the to_set method.
Set uses Hash as storage, so you must note the following points:
Equality of elements is determined according to Object#eql? and Object#hash. Use Set#compare_by_identity to make a set compare its elements by their identity.
Set assumes that the identity of each element does not change while it is stored. Modifying an element of a set will render the set to an unreliable state.
When a string is to be stored, a frozen copy of the string is stored instead unless the original string is already frozen.
The comparison operators <, >, <= and >= are implemented as shorthand for the {proper_,}{subset?,superset?} methods. However, the <=> operator is intentionally left out because not every pair of sets is comparable. ({x,y} vs. {x,z} for example)
require 'set' s1 = Set.new [1, 2] # -> #<Set: {1, 2}> s2 = [1, 2].to_set # -> #<Set: {1, 2}> s1 == s2 # -> true s1.add("foo") # -> #<Set: {1, 2, "foo"}> s1.merge([2, 6]) # -> #<Set: {1, 2, "foo", 6}> s1.subset? s2 # -> false s2.subset? s1 # -> true
- Akinori MUSHA <knu@iDaemons.org> (current maintainer)
SortedSet implements a Set that guarantees that its elements are yielded in sorted order (according to the return values of their <=> methods) when iterating over them.
All elements that are added to a SortedSet must respond to the <=> method for comparison.
Also, all elements must be mutually comparable: el1 <=> el2 must not return nil for any elements el1 and el2, else an ArgumentError will be raised when iterating over the SortedSet.
require "set" set = SortedSet.new([2, 1, 5, 6, 4, 5, 3, 3, 3]) ary = [] set.each do |obj| ary << obj end p ary # => [1, 2, 3, 4, 5, 6] set2 = SortedSet.new([1, 2, "3"]) set2.each { |obj| } # => raises ArgumentError: comparison of Fixnum with String failed
Raised when attempting to convert special float values (in particular infinite or NaN) to numerical classes which don’t support them.
Float::INFINITY.to_r #=> FloatDomainError: Infinity
The global value false is the only instance of class FalseClass and represents a logically false value in boolean expressions. The class provides operators allowing false to participate correctly in logical expressions.
Raised when Ruby can’t yield as requested.
A typical scenario is attempting to yield when no block is given:
def call_block yield 42 end call_block
raises the exception:
LocalJumpError: no block given (yield)
A more subtle example:
def get_me_a_return Proc.new { return 42 } end get_me_a_return.call
raises the exception:
LocalJumpError: unexpected return
Raised in case of a stack overflow.
def me_myself_and_i me_myself_and_i end me_myself_and_i
raises the exception:
SystemStackError: stack level too deep
Raised when given an invalid regexp expression.
Regexp.new("?")
raises the exception:
RegexpError: target of repeat operator is not specified: /?/
Raised when an invalid operation is attempted on a thread.
For example, when no other thread has been started:
Thread.stop
This will raises the following exception:
ThreadError: stopping only thread note: use sleep to stop forever