Overview

Psych is a YAML parser and emitter. Psych leverages libyaml [Home page: pyyaml.org/wiki/LibYAML] or [git repo: github.com/yaml/libyaml] for its YAML parsing and emitting capabilities. In addition to wrapping libyaml, Psych also knows how to serialize and de-serialize most Ruby objects to and from the YAML format.

I NEED TO PARSE OR EMIT YAML RIGHT NOW!

# Parse some YAML
Psych.load("--- foo") # => "foo"

# Emit some YAML
Psych.dump("foo")     # => "--- foo\n...\n"
{ :a => 'b'}.to_yaml  # => "---\n:a: b\n"

Got more time on your hands? Keep on reading!

YAML Parsing

Psych provides a range of interfaces for parsing a YAML document ranging from low level to high level, depending on your parsing needs. At the lowest level, is an event based parser. Mid level is access to the raw YAML AST, and at the highest level is the ability to unmarshal YAML to Ruby objects.

YAML Emitting

Psych provides a range of interfaces ranging from low to high level for producing YAML documents. Very similar to the YAML parsing interfaces, Psych provides at the lowest level, an event based system, mid-level is building a YAML AST, and the highest level is converting a Ruby object straight to a YAML document.

High-level API

Parsing

The high level YAML parser provided by Psych simply takes YAML as input and returns a Ruby data structure. For information on using the high level parser see Psych.load

Reading from a string

Psych.safe_load("--- a")             # => 'a'
Psych.safe_load("---\n - a\n - b")   # => ['a', 'b']
# From a trusted string:
Psych.load("--- !ruby/range\nbegin: 0\nend: 42\nexcl: false\n") # => 0..42

Reading from a file

Psych.safe_load_file("data.yml", permitted_classes: [Date])
Psych.load_file("trusted_database.yml")

Exception handling

begin
  # The second argument changes only the exception contents
  Psych.parse("--- `", "file.txt")
rescue Psych::SyntaxError => ex
  ex.file    # => 'file.txt'
  ex.message # => "(file.txt): found character that cannot start any token"
end

Emitting

The high level emitter has the easiest interface. Psych simply takes a Ruby data structure and converts it to a YAML document. See Psych.dump for more information on dumping a Ruby data structure.

Writing to a string

# Dump an array, get back a YAML string
Psych.dump(['a', 'b'])  # => "---\n- a\n- b\n"

# Dump an array to an IO object
Psych.dump(['a', 'b'], StringIO.new)  # => #<StringIO:0x000001009d0890>

# Dump an array with indentation set
Psych.dump(['a', ['b']], :indentation => 3) # => "---\n- a\n-  - b\n"

# Dump an array to an IO with indentation set
Psych.dump(['a', ['b']], StringIO.new, :indentation => 3)

Writing to a file

Currently there is no direct API for dumping Ruby structure to file:

File.open('database.yml', 'w') do |file|
  file.write(Psych.dump(['a', 'b']))
end

Mid-level API

Parsing

Psych provides access to an AST produced from parsing a YAML document. This tree is built using the Psych::Parser and Psych::TreeBuilder. The AST can be examined and manipulated freely. Please see Psych::parse_stream, Psych::Nodes, and Psych::Nodes::Node for more information on dealing with YAML syntax trees.

Reading from a string

# Returns Psych::Nodes::Stream
Psych.parse_stream("---\n - a\n - b")

# Returns Psych::Nodes::Document
Psych.parse("---\n - a\n - b")

Reading from a file

# Returns Psych::Nodes::Stream
Psych.parse_stream(File.read('database.yml'))

# Returns Psych::Nodes::Document
Psych.parse_file('database.yml')

Exception handling

begin
  # The second argument changes only the exception contents
  Psych.parse("--- `", "file.txt")
rescue Psych::SyntaxError => ex
  ex.file    # => 'file.txt'
  ex.message # => "(file.txt): found character that cannot start any token"
end

Emitting

At the mid level is building an AST. This AST is exactly the same as the AST used when parsing a YAML document. Users can build an AST by hand and the AST knows how to emit itself as a YAML document. See Psych::Nodes, Psych::Nodes::Node, and Psych::TreeBuilder for more information on building a YAML AST.

Writing to a string

# We need Psych::Nodes::Stream (not Psych::Nodes::Document)
stream = Psych.parse_stream("---\n - a\n - b")

stream.to_yaml # => "---\n- a\n- b\n"

Writing to a file

# We need Psych::Nodes::Stream (not Psych::Nodes::Document)
stream = Psych.parse_stream(File.read('database.yml'))

File.open('database.yml', 'w') do |file|
  file.write(stream.to_yaml)
end

Low-level API

Parsing

The lowest level parser should be used when the YAML input is already known, and the developer does not want to pay the price of building an AST or automatic detection and conversion to Ruby objects. See Psych::Parser for more information on using the event based parser.

Reading to Psych::Nodes::Stream structure

parser = Psych::Parser.new(TreeBuilder.new) # => #<Psych::Parser>
parser = Psych.parser                       # it's an alias for the above

parser.parse("---\n - a\n - b")             # => #<Psych::Parser>
parser.handler                              # => #<Psych::TreeBuilder>
parser.handler.root                         # => #<Psych::Nodes::Stream>

Receiving an events stream

recorder = Psych::Handlers::Recorder.new
parser = Psych::Parser.new(recorder)

parser.parse("---\n - a\n - b")
recorder.events # => [list of [event, args] lists]
                # event is one of: Psych::Handler::EVENTS
                # args are the arguments passed to the event

Emitting

The lowest level emitter is an event based system. Events are sent to a Psych::Emitter object. That object knows how to convert the events to a YAML document. This interface should be used when document format is known in advance or speed is a concern. See Psych::Emitter for more information.

Writing to a Ruby structure

Psych.parser.parse("--- a")       # => #<Psych::Parser>

parser.handler.first              # => #<Psych::Nodes::Stream>
parser.handler.first.to_ruby      # => ["a"]

parser.handler.root.first         # => #<Psych::Nodes::Document>
parser.handler.root.first.to_ruby # => "a"

# You can instantiate an Emitter manually
Psych::Visitors::ToRuby.new.accept(parser.handler.root.first)
# => "a"

define UnicodeNormalize module here so that we don’t have to look it up

The marshaling library converts collections of Ruby objects into a byte stream, allowing them to be stored outside the currently active script. This data may subsequently be read and the original objects reconstituted.

Marshaled data has major and minor version numbers stored along with the object information. In normal use, marshaling can only load data written with the same major version number and an equal or lower minor version number. If Ruby’s “verbose” flag is set (normally using -d, -v, -w, or –verbose) the major and minor numbers must match exactly. Marshal versioning is independent of Ruby’s version numbers. You can extract the version by reading the first two bytes of marshaled data.

str = Marshal.dump("thing")
RUBY_VERSION   #=> "1.9.0"
str[0].ord     #=> 4
str[1].ord     #=> 8

Some objects cannot be dumped: if the objects to be dumped include bindings, procedure or method objects, instances of class IO, or singleton objects, a TypeError will be raised.

If your class has special serialization needs (for example, if you want to serialize in some specific format), or if it contains objects that would otherwise not be serializable, you can implement your own serialization strategy.

There are two methods of doing this, your object can define either marshal_dump and marshal_load or _dump and _load. marshal_dump will take precedence over _dump if both are defined. marshal_dump may result in smaller Marshal strings.

Security considerations

By design, Marshal.load can deserialize almost any class loaded into the Ruby process. In many cases this can lead to remote code execution if the Marshal data is loaded from an untrusted source.

As a result, Marshal.load is not suitable as a general purpose serialization format and you should never unmarshal user supplied input or other untrusted data.

If you need to deserialize untrusted data, use JSON or another serialization format that is only able to load simple, ‘primitive’ types such as String, Array, Hash, etc. Never allow user input to specify arbitrary types to deserialize into.

marshal_dump and marshal_load

When dumping an object the method marshal_dump will be called. marshal_dump must return a result containing the information necessary for marshal_load to reconstitute the object. The result can be any object.

When loading an object dumped using marshal_dump the object is first allocated then marshal_load is called with the result from marshal_dump. marshal_load must recreate the object from the information in the result.

Example:

class MyObj
  def initialize name, version, data
    @name    = name
    @version = version
    @data    = data
  end

  def marshal_dump
    [@name, @version]
  end

  def marshal_load array
    @name, @version = array
  end
end

_dump and _load

Use _dump and _load when you need to allocate the object you’re restoring yourself.

When dumping an object the instance method _dump is called with an Integer which indicates the maximum depth of objects to dump (a value of -1 implies that you should disable depth checking). _dump must return a String containing the information necessary to reconstitute the object.

The class method _load should take a String and use it to return an object of the same class.

Example:

class MyObj
  def initialize name, version, data
    @name    = name
    @version = version
    @data    = data
  end

  def _dump level
    [@name, @version].join ':'
  end

  def self._load args
    new(*args.split(':'))
  end
end

Since Marshal.dump outputs a string you can have _dump return a Marshal string which is Marshal.loaded in _load for complex objects.

Class for representing HTTP method PATCH:

require 'net/http'
uri = URI('http://example.com')
hostname = uri.hostname # => "example.com"
uri.path = '/posts'
req = Net::HTTP::Patch.new(uri) # => #<Net::HTTP::Patch PATCH>
req.body = '{"title": "foo","body": "bar","userId": 1}'
req.content_type = 'application/json'
res = Net::HTTP.start(hostname) do |http|
  http.request(req)
end

Properties:

• Request body: yes.

• Response body: yes.

• Safe: no.

• Idempotent: no.

• Cacheable: no.

Related:

• Net::HTTP#patch: sends PATCH request, returns response object.

Class for representing WebDAV method PROPPATCH:

require 'net/http'
uri = URI('http://example.com')
hostname = uri.hostname # => "example.com"
req = Net::HTTP::Proppatch.new(uri) # => #<Net::HTTP::Proppatch PROPPATCH>
res = Net::HTTP.start(hostname) do |http|
  http.request(req)
end

Related:

• Net::HTTP#proppatch: sends PROPPATCH request, returns response object.

Generator

Enumerator::Chain is a subclass of Enumerator, which represents a chain of enumerables that works as a single enumerator.

This type of objects can be created by Enumerable#chain and Enumerator#+.

Raised by Encoding and String methods when the source encoding is incompatible with the target encoding.

This exception is raised if a generator or unparser error occurs.

OpenSSL::HMAC allows computing Hash-based Message Authentication Code (HMAC). It is a type of message authentication code (MAC) involving a hash function in combination with a key. HMAC can be used to verify the integrity of a message as well as the authenticity.

OpenSSL::HMAC has a similar interface to OpenSSL::Digest.

HMAC-SHA256 using one-shot interface

key = "key"
data = "message-to-be-authenticated"
mac = OpenSSL::HMAC.hexdigest("SHA256", key, data)
#=> "cddb0db23f469c8bf072b21fd837149bd6ace9ab771cceef14c9e517cc93282e"

HMAC-SHA256 using incremental interface

data1 = File.binread("file1")
data2 = File.binread("file2")
key = "key"
hmac = OpenSSL::HMAC.new(key, 'SHA256')
hmac << data1
hmac << data2
mac = hmac.digest

Document-class: OpenSSL::HMAC

OpenSSL::HMAC allows computing Hash-based Message Authentication Code (HMAC). It is a type of message authentication code (MAC) involving a hash function in combination with a key. HMAC can be used to verify the integrity of a message as well as the authenticity.

OpenSSL::HMAC has a similar interface to OpenSSL::Digest.

HMAC-SHA256 using one-shot interface

key = "key"
data = "message-to-be-authenticated"
mac = OpenSSL::HMAC.hexdigest("SHA256", key, data)
#=> "cddb0db23f469c8bf072b21fd837149bd6ace9ab771cceef14c9e517cc93282e"

HMAC-SHA256 using incremental interface

data1 = File.binread("file1")
data2 = File.binread("file2")
key = "key"
hmac = OpenSSL::HMAC.new(key, 'SHA256')
hmac << data1
hmac << data2
mac = hmac.digest

Subclasses ‘BadAlias` for backwards compatibility

Thrown when PTY::check is called for a pid that represents a process that has exited.

Socket::AncillaryData represents the ancillary data (control information) used by sendmsg and recvmsg system call. It contains socket family, control message (cmsg) level, cmsg type and cmsg data.

Subclass of Zlib::Error when zlib returns a Z_DATA_ERROR.

Usually if a stream was prematurely freed.

Zlib::Deflate is the class for compressing data. See Zlib::ZStream for more information.

Zlib:Inflate is the class for decompressing compressed data. Unlike Zlib::Deflate, an instance of this class is not able to duplicate (clone, dup) itself.

Objects of class File::Stat encapsulate common status information for File objects. The information is recorded at the moment the File::Stat object is created; changes made to the file after that point will not be reflected. File::Stat objects are returned by IO#stat, File::stat, File#lstat, and File::lstat. Many of these methods return platform-specific values, and not all values are meaningful on all systems. See also Kernel#test.

An ObjectSpace::WeakMap object holds references to any objects, but those objects can get garbage collected.

This class is mostly used internally by WeakRef, please use lib/weakref.rb for the public interface.

The error thrown when the parser encounters illegal CSV formatting.