Net

Module
Constants

HTTPSession

Class Net::HTTP provides a rich library that implements the client in a client-server model that uses the HTTP request-response protocol. For information about HTTP, see:

About the Examples

Strategies

  • If you will make only a few GET requests, consider using OpenURI.

  • If you will make only a few requests of all kinds, consider using the various singleton convenience methods in this class. Each of the following methods automatically starts and finishes a session that sends a single request:

    # Return string response body.
Net::HTTP.get(hostname, path)
Net::HTTP.get(uri)

# Write string response body to $stdout.
Net::HTTP.get_print(hostname, path)
Net::HTTP.get_print(uri)

# Return response as Net::HTTPResponse object.
Net::HTTP.get_response(hostname, path)
Net::HTTP.get_response(uri)
data = '{"title": "foo", "body": "bar", "userId": 1}'
Net::HTTP.post(uri, data)
params = {title: 'foo', body: 'bar', userId: 1}
Net::HTTP.post_form(uri, params)
data = '{"title": "foo", "body": "bar", "userId": 1}'
Net::HTTP.put(uri, data)

  • If performance is important, consider using sessions, which lower request overhead. This session has multiple requests for HTTP methods and WebDAV methods:

    Net::HTTP.start(hostname) do |http|
  # Session started automatically before block execution.
  http.get(path)
  http.head(path)
  body = 'Some text'
  http.post(path, body)  # Can also have a block.
  http.put(path, body)
  http.delete(path)
  http.options(path)
  http.trace(path)
  http.patch(path, body) # Can also have a block.
  http.copy(path)
  http.lock(path, body)
  http.mkcol(path, body)
  http.move(path)
  http.propfind(path, body)
  http.proppatch(path, body)
  http.unlock(path, body)
  # Session finished automatically at block exit.
end

The methods cited above are convenience methods that, via their few arguments, allow minimal control over the requests. For greater control, consider using request objects.

URIs

On the internet, a URI (Universal Resource Identifier) is a string that identifies a particular resource. It consists of some or all of: scheme, hostname, path, query, and fragment; see URI syntax.

A Ruby URI::Generic object represents an internet URI. It provides, among others, methods scheme, hostname, path, query, and fragment.

Schemes

An internet URI has a scheme.

The two schemes supported in Net::HTTP are 'https' and 'http':

uri.scheme                       # => "https"
URI('http://example.com').scheme # => "http"

Hostnames

A hostname identifies a server (host) to which requests may be sent:

hostname = uri.hostname # => "jsonplaceholder.typicode.com"
Net::HTTP.start(hostname) do |http|
  # Some HTTP stuff.
end

Paths

A host-specific path identifies a resource on the host:

_uri = uri.dup
_uri.path = '/todos/1'
hostname = _uri.hostname
path = _uri.path
Net::HTTP.get(hostname, path)

Queries

A host-specific query adds name/value pairs to the URI:

_uri = uri.dup
params = {userId: 1, completed: false}
_uri.query = URI.encode_www_form(params)
_uri # => #<URI::HTTPS https://jsonplaceholder.typicode.com?userId=1&completed=false>
Net::HTTP.get(_uri)

Fragments

A URI fragment has no effect in Net::HTTP; the same data is returned, regardless of whether a fragment is included.

Request Headers

Request headers may be used to pass additional information to the host, similar to arguments passed in a method call; each header is a name/value pair.

Each of the Net::HTTP methods that sends a request to the host has optional argument headers, where the headers are expressed as a hash of field-name/value pairs:

headers = {Accept: 'application/json', Connection: 'Keep-Alive'}
Net::HTTP.get(uri, headers)

See lists of both standard request fields and common request fields at Request Fields. A host may also accept other custom fields.

HTTP Sessions

A session is a connection between a server (host) and a client that:

See example sessions at Strategies.

Session Using Net::HTTP.start

If you have many requests to make to a single host (and port), consider using singleton method Net::HTTP.start with a block; the method handles the session automatically by:

  • Calling start before block execution.

  • Executing the block.

  • Calling finish after block execution.

In the block, you can use these instance methods, each of which that sends a single request:

  • HTTP methods:

    • get, request_get: GET.

    • head, request_head: HEAD.

    • post, request_post: POST.

    • delete: DELETE.

    • options: OPTIONS.

    • trace: TRACE.

    • patch: PATCH.

  • WebDAV methods:

    • copy: COPY.

    • lock: LOCK.

    • mkcol: MKCOL.

    • move: MOVE.

    • propfind: PROPFIND.

    • proppatch: PROPPATCH.

    • unlock: UNLOCK.

Session Using Net::HTTP.start and Net::HTTP.finish

You can manage a session manually using methods start and finish:

http = Net::HTTP.new(hostname)
http.start
http.get('/todos/1')
http.get('/todos/2')
http.delete('/posts/1')
http.finish # Needed to free resources.

Single-Request Session

Certain convenience methods automatically handle a session by:

  • Creating an HTTP object

  • Starting a session.

  • Sending a single request.

  • Finishing the session.

  • Destroying the object.

Such methods that send GET requests:

  • ::get: Returns the string response body.

  • ::get_print: Writes the string response body to $stdout.

  • ::get_response: Returns a Net::HTTPResponse object.

Such methods that send POST requests:

  • ::post: Posts data to the host.

  • ::post_form: Posts form data to the host.

HTTP Requests and Responses

Many of the methods above are convenience methods, each of which sends a request and returns a string without directly using Net::HTTPRequest and Net::HTTPResponse objects.

You can, however, directly create a request object, send the request, and retrieve the response object; see:

Following Redirection

Each returned response is an instance of a subclass of Net::HTTPResponse. See the response class hierarchy.

In particular, class Net::HTTPRedirection is the parent of all redirection classes. This allows you to craft a case statement to handle redirections properly:

def fetch(uri, limit = 10)
  # You should choose a better exception.
  raise ArgumentError, 'Too many HTTP redirects' if limit == 0

  res = Net::HTTP.get_response(URI(uri))
  case res
  when Net::HTTPSuccess     # Any success class.
    res
  when Net::HTTPRedirection # Any redirection class.
    location = res['Location']
    warn "Redirected to #{location}"
    fetch(location, limit - 1)
  else                      # Any other class.
    res.value
  end
end

fetch(uri)

Basic Authentication

Basic authentication is performed according to RFC2617:

req = Net::HTTP::Get.new(uri)
req.basic_auth('user', 'pass')
res = Net::HTTP.start(hostname) do |http|
  http.request(req)
end

Streaming Response Bodies

By default Net::HTTP reads an entire response into memory. If you are handling large files or wish to implement a progress bar you can instead stream the body directly to an IO.

Net::HTTP.start(hostname) do |http|
  req = Net::HTTP::Get.new(uri)
  http.request(req) do |res|
    open('t.tmp', 'w') do |f|
      res.read_body do |chunk|
        f.write chunk
      end
    end
  end
end

HTTPS

HTTPS is enabled for an HTTP connection by Net::HTTP#use_ssl=:

Net::HTTP.start(hostname, :use_ssl => true) do |http|
  req = Net::HTTP::Get.new(uri)
  res = http.request(req)
end

Or if you simply want to make a GET request, you may pass in a URI object that has an HTTPS URL. Net::HTTP automatically turns on TLS verification if the URI object has a ‘https’ URI scheme:

uri # => #<URI::HTTPS https://jsonplaceholder.typicode.com/>
Net::HTTP.get(uri)

Proxy Server

An HTTP object can have a proxy server.

You can create an HTTP object with a proxy server using method Net::HTTP.new or method Net::HTTP.start.

The proxy may be defined either by argument p_addr or by environment variable 'http_proxy'.

Proxy Using Argument p_addr as a String

When argument p_addr is a string hostname, the returned http has the given host as its proxy:

http = Net::HTTP.new(hostname, nil, 'proxy.example')
http.proxy?          # => true
http.proxy_from_env? # => false
http.proxy_address   # => "proxy.example"
# These use default values.
http.proxy_port      # => 80
http.proxy_user      # => nil
http.proxy_pass      # => nil

The port, username, and password for the proxy may also be given:

http = Net::HTTP.new(hostname, nil, 'proxy.example', 8000, 'pname', 'ppass')
# => #<Net::HTTP jsonplaceholder.typicode.com:80 open=false>
http.proxy?          # => true
http.proxy_from_env? # => false
http.proxy_address   # => "proxy.example"
http.proxy_port      # => 8000
http.proxy_user      # => "pname"
http.proxy_pass      # => "ppass"

Proxy Using ‘ENV['http_proxy']’

When environment variable 'http_proxy' is set to a URI string, the returned http will have the server at that URI as its proxy; note that the URI string must have a protocol such as 'http' or 'https':

ENV['http_proxy'] = 'http://example.com'
http = Net::HTTP.new(hostname)
http.proxy?          # => true
http.proxy_from_env? # => true
http.proxy_address   # => "example.com"
# These use default values.
http.proxy_port      # => 80
http.proxy_user      # => nil
http.proxy_pass      # => nil

The URI string may include proxy username, password, and port number:

ENV['http_proxy'] = 'http://pname:ppass@example.com:8000'
http = Net::HTTP.new(hostname)
http.proxy?          # => true
http.proxy_from_env? # => true
http.proxy_address   # => "example.com"
http.proxy_port      # => 8000
http.proxy_user      # => "pname"
http.proxy_pass      # => "ppass"

Filtering Proxies

With method Net::HTTP.new (but not Net::HTTP.start), you can use argument p_no_proxy to filter proxies:

  • Reject a certain address:

    http = Net::HTTP.new('example.com', nil, 'proxy.example', 8000, 'pname', 'ppass', 'proxy.example')
http.proxy_address # => nil

  • Reject certain domains or subdomains:

    http = Net::HTTP.new('example.com', nil, 'my.proxy.example', 8000, 'pname', 'ppass', 'proxy.example')
http.proxy_address # => nil

  • Reject certain addresses and port combinations:

    http = Net::HTTP.new('example.com', nil, 'proxy.example', 8000, 'pname', 'ppass', 'proxy.example:1234')
http.proxy_address # => "proxy.example"

http = Net::HTTP.new('example.com', nil, 'proxy.example', 8000, 'pname', 'ppass', 'proxy.example:8000')
http.proxy_address # => nil

  • Reject a list of the types above delimited using a comma:

    http = Net::HTTP.new('example.com', nil, 'proxy.example', 8000, 'pname', 'ppass', 'my.proxy,proxy.example:8000')
http.proxy_address # => nil

http = Net::HTTP.new('example.com', nil, 'my.proxy', 8000, 'pname', 'ppass', 'my.proxy,proxy.example:8000')
http.proxy_address # => nil

Compression and Decompression

Net::HTTP does not compress the body of a request before sending.

By default, Net::HTTP adds header 'Accept-Encoding' to a new request object:

Net::HTTP::Get.new(uri)['Accept-Encoding']
# => "gzip;q=1.0,deflate;q=0.6,identity;q=0.3"

This requests the server to zip-encode the response body if there is one; the server is not required to do so.

Net::HTTP does not automatically decompress a response body if the response has header 'Content-Range'.

Otherwise decompression (or not) depends on the value of header Content-Encoding:

  • 'deflate', 'gzip', or 'x-gzip': decompresses the body and deletes the header.

  • 'none' or 'identity': does not decompress the body, but deletes the header.

  • Any other value: leaves the body and header unchanged.

What’s Here

First, what’s elsewhere. Class Net::HTTP:

This is a categorized summary of methods and attributes.

Net::HTTP Objects

  • ::new: Creates a new instance.

  • #inspect: Returns a string representation of self.

Sessions

  • ::start: Begins a new session in a new Net::HTTP object.

  • #started?: Returns whether in a session.

  • #finish: Ends an active session.

  • #start: Begins a new session in an existing Net::HTTP object (self).

Connections

Requests

  • ::get: Sends a GET request and returns the string response body.

  • ::get_print: Sends a GET request and write the string response body to $stdout.

  • ::get_response: Sends a GET request and returns a response object.

  • ::post_form: Sends a POST request with form data and returns a response object.

  • ::post: Sends a POST request with data and returns a response object.

  • ::put: Sends a PUT request with data and returns a response object.

  • #copy: Sends a COPY request and returns a response object.

  • #delete: Sends a DELETE request and returns a response object.

  • #get: Sends a GET request and returns a response object.

  • #head: Sends a HEAD request and returns a response object.

  • #lock: Sends a LOCK request and returns a response object.

  • #mkcol: Sends a MKCOL request and returns a response object.

  • #move: Sends a MOVE request and returns a response object.

  • #options: Sends a OPTIONS request and returns a response object.

  • #patch: Sends a PATCH request and returns a response object.

  • #post: Sends a POST request and returns a response object.

  • #propfind: Sends a PROPFIND request and returns a response object.

  • #proppatch: Sends a PROPPATCH request and returns a response object.

  • #put: Sends a PUT request and returns a response object.

  • #request: Sends a request and returns a response object.

  • #request_get: Sends a GET request and forms a response object; if a block given, calls the block with the object, otherwise returns the object.

  • #request_head: Sends a HEAD request and forms a response object; if a block given, calls the block with the object, otherwise returns the object.

  • #request_post: Sends a POST request and forms a response object; if a block given, calls the block with the object, otherwise returns the object.

  • #send_request: Sends a request and returns a response object.

  • #trace: Sends a TRACE request and returns a response object.

  • #unlock: Sends an UNLOCK request and returns a response object.

Responses

Proxies

Security

Addresses and Ports

  • :address: Returns the string host name or host IP.

  • ::default_port: Returns integer 80, the default port to use for HTTP requests.

  • ::http_default_port: Returns integer 80, the default port to use for HTTP requests.

  • ::https_default_port: Returns integer 443, the default port to use for HTTPS requests.

  • #ipaddr: Returns the IP address for the connection.

  • #ipaddr=: Sets the IP address for the connection.

  • :local_host: Returns the string local host used to establish the connection.

  • :local_host=: Sets the string local host used to establish the connection.

  • :local_port: Returns the integer local port used to establish the connection.

  • :local_port=: Sets the integer local port used to establish the connection.

  • :port: Returns the integer port number.

HTTP Version

Debugging

HTTPServerException

We cannot use the name “HTTPServerError”, it is the name of the response.

HTTPMultipleChoice

Response class for Multiple Choices responses (status code 300).

The Multiple Choices response indicates that the server offers multiple options for the resource from which the client may choose.

References:

HTTPMovedTemporarily

Response class for Found responses (status code 302).

The Found response indicates that the client should look at (browse to) another URL.

References:

HTTPRequestTimeOut

Response class for Request Timeout responses (status code 408).

The server timed out waiting for the request.

References:

HTTPRequestEntityTooLarge

Response class for Payload Too Large responses (status code 413).

The request is larger than the server is willing or able to process.

References:

HTTPRequestURITooLong

Response class for URI Too Long responses (status code 414).

The URI provided was too long for the server to process.

References:

HTTPRequestURITooLarge

Response class for URI Too Long responses (status code 414).

The URI provided was too long for the server to process.

References:

HTTPRequestedRangeNotSatisfiable

Response class for Range Not Satisfiable responses (status code 416).

The request entity has a media type which the server or resource does not support.

References:

HTTPGatewayTimeOut

Response class for Gateway Timeout responses (status code 504).

The server was acting as a gateway or proxy and did not receive a timely response from the upstream server.

References: