Acts like a StringIO
with reduced API, but without having to require that class.
Outputs code with highlighted lines
Whatever is passed to this class will be rendered even if it is “marked invisible” any filtering of output should be done before calling this class.
DisplayCodeWithLineNumbers.new( lines: lines, highlight_lines: [lines[2], lines[3]] ).call # => 1 2 def cat > 3 Dir.chdir > 4 end 5 end 6
Converts a SyntaxError
message to a path
Handles the case where the filename has a colon in it such as on a windows file system: github.com/ruby/syntax_suggest/issues/111
Example:
message = "/tmp/scratch:2:in `require_relative': /private/tmp/bad.rb:1: syntax error, unexpected `end' (SyntaxError)" puts PathnameFromMessage.new(message).call.name # => "/tmp/scratch.rb"
Tracks which lines various code blocks have expanded to and which are still unexplored
Not a URI
component.
Module File::Constants
defines file-related constants.
There are two families of constants here:
Those having to do with file access.
Those having to do with filename globbing.
File constants defined for the local process may be retrieved with method File::Constants.constants:
File::Constants.constants.take(5) # => [:RDONLY, :WRONLY, :RDWR, :APPEND, :CREAT]
File-access constants may be used with optional argument mode
in calls to the following methods:
Read-write access for a stream may be specified by a file-access constant.
The constant may be specified as part of a bitwise OR of other such constants.
Any combination of the constants in this section may be specified.
Flag File::RDONLY specifies the stream should be opened for reading only:
filepath = '/tmp/t.tmp' f = File.new(filepath, File::RDONLY) f.write('Foo') # Raises IOError (not opened for writing).
Flag File::WRONLY specifies that the stream should be opened for writing only:
f = File.new(filepath, File::WRONLY) f.read # Raises IOError (not opened for reading).
Flag File::RDWR specifies that the stream should be opened for both reading and writing:
f = File.new(filepath, File::RDWR) f.write('Foo') # => 3 f.rewind # => 0 f.read # => "Foo"
Flag File::APPEND specifies that the stream should be opened in append mode.
Before each write operation, the position is set to end-of-stream. The modification of the position and the following write operation are performed as a single atomic step.
Flag File::TRUNC specifies that the stream should be truncated at its beginning. If the file exists and is successfully opened for writing, it is to be truncated to position zero; its ctime and mtime are updated.
There is no effect on a FIFO special file or a terminal device. The effect on other file types is implementation-defined. The result of using File::TRUNC with File::RDONLY is undefined.
Flag File::CREAT specifies that the stream should be created if it does not already exist.
If the file exists:
- Raise an exception if File::EXCL is also specified. - Otherwise, do nothing.
If the file does not exist, then it is created. Upon successful completion, the atime, ctime, and mtime of the file are updated, and the ctime and mtime of the parent directory are updated.
Flag File::EXCL specifies that the stream should not already exist; If flags File::CREAT and File::EXCL are both specified and the stream already exists, an exception is raised.
The check for the existence and creation of the file is performed as an atomic operation.
If both File::EXCL and File::CREAT are specified and the path names a symbolic link, an exception is raised regardless of the contents of the symbolic link.
If File::EXCL is specified and File::CREAT is not specified, the result is undefined.
Some file-access constants are defined only on POSIX-compliant systems; those are:
File::SYNC.
File::DSYNC.
File::RSYNC.
File::DIRECT.
File::NOATIME.
File::NOCTTY.
File::NOFOLLOW.
File::TMPFILE.
Flag File::SYNC, File::RSYNC, or File::DSYNC specifies synchronization of I/O operations with the underlying file system.
These flags are valid only for POSIX-compliant systems.
File::SYNC specifies that all write operations (both data and metadata) are immediately to be flushed to the underlying storage device. This means that the data is written to the storage device, and the file’s metadata (e.g., file size, timestamps, permissions) are also synchronized. This guarantees that data is safely stored on the storage medium before returning control to the calling program. This flag can have a significant impact on performance since it requires synchronous writes, which can be slower compared to asynchronous writes.
File::RSYNC specifies that any read operations on the file will not return until all outstanding write operations (those that have been issued but not completed) are also synchronized. This is useful when you want to read the most up-to-date data, which may still be in the process of being written.
File::DSYNC specifies that all data write operations are immediately to be flushed to the underlying storage device; this differs from File::SYNC, which requires that metadata also be synchronized.
Note that the behavior of these flags may vary slightly depending on the operating system and filesystem being used. Additionally, using these flags can have an impact on performance due to the synchronous nature of the I/O operations, so they should be used judiciously, especially in performance-critical applications.
Flag File::NOCTTY specifies that if the stream is a terminal device, that device does not become the controlling terminal for the process.
Defined only for POSIX-compliant systems.
Flag File::DIRECT requests that cache effects of the I/O to and from the stream be minimized.
Defined only for POSIX-compliant systems.
Flag File::NOATIME specifies that act of opening the stream should not modify its access time (atime).
Defined only for POSIX-compliant systems.
Flag File::NOFOLLOW specifies that if path is a symbolic link, it should not be followed.
Defined only for POSIX-compliant systems.
Flag File::TMPFILE specifies that the opened stream should be a new temporary file.
Defined only for POSIX-compliant systems.
When possible, the file is opened in nonblocking mode. Neither the open operation nor any subsequent I/O operations on the file will cause the calling process to wait.
Flag File::BINARY specifies that the stream is to be accessed in binary mode.
Flag File::SHARE_DELETE enables other processes to open the stream with delete access.
Windows only.
If the stream is opened for (local) delete access without File::SHARE_DELETE, and another process attempts to open it with delete access, the attempt fails and the stream is not opened for that process.
Four file constants relate to stream locking; see File#flock
:
Flag File::LOCK_EX specifies an exclusive lock; only one process a a time may lock the stream.
Flag File::LOCK_NB specifies non-blocking locking for the stream; may be combined with File::LOCK_EX or File::LOCK_SH.
Flag File::LOCK_SH specifies that multiple processes may lock the stream at the same time.
Flag File::LOCK_UN specifies that the stream is not to be locked.
Filename-globbing constants may be used with optional argument flags
in calls to the following methods:
The constants are:
Flag File::FNM_CASEFOLD makes patterns case insensitive for File.fnmatch
(but not Dir.glob
).
Flag File::FNM_DOTMATCH makes the '*'
pattern match a filename starting with '.'
.
Flag File::FNM_EXTGLOB enables pattern '{a,b}'
, which matches pattern ‘a’ and pattern ‘b’; behaves like a regexp union (e.g., '(?:a|b)'
):
pattern = '{LEGAL,BSDL}' Dir.glob(pattern) # => ["LEGAL", "BSDL"] Pathname.glob(pattern) # => [#<Pathname:LEGAL>, #<Pathname:BSDL>] pathname.glob(pattern) # => [#<Pathname:LEGAL>, #<Pathname:BSDL>]
Flag File::FNM_NOESCAPE disables '\'
escaping.
Flag File::FNM_PATHNAME specifies that patterns '*'
and '?'
do not match the directory separator (the value of constant File::SEPARATOR).
Flag File::FNM_SHORTNAME allows patterns to match short names if they exist.
Windows only.
Flag File::FNM_SYSCASE specifies that case sensitivity is the same as in the underlying operating system; effective for File.fnmatch
, but not Dir.glob
.
Flag File::NULL contains the string value of the null device:
On a Unix-like OS, '/dev/null'
.
On Windows, 'NUL'
.
A DSL that provides the means to dynamically load libraries and build modules around them including calling extern functions within the C library that has been loaded.
require 'fiddle' require 'fiddle/import' module LibSum extend Fiddle::Importer dlload './libsum.so' extern 'double sum(double*, int)' extern 'double split(double)' end
exception to wait for reading. see IO.select
.
The WIN32OLE::VariantType
module includes constants of VARIANT type constants. The constants is used when creating WIN32OLE::Variant
object.
obj = WIN32OLE::Variant.new("2e3", WIN32OLE::VARIANT::VT_R4) obj.value # => 2000.0
This module has all methods of FileUtils
module, but never changes files/directories. This equates to passing the :noop
flag to methods in FileUtils
.
Logging severity.
Extends command line arguments array (ARGV) to parse itself.
A parser for the pack template language.
The Gem::Security
implements cryptographic signatures for gems. The section below is a step-by-step guide to using signed gems and generating your own.
In order to start signing your gems, you’ll need to build a private key and a self-signed certificate. Here’s how:
# build a private key and certificate for yourself: $ gem cert --build you@example.com
This could take anywhere from a few seconds to a minute or two, depending on the speed of your computer (public key algorithms aren’t exactly the speediest crypto algorithms in the world). When it’s finished, you’ll see the files “gem-private_key.pem” and “gem-public_cert.pem” in the current directory.
First things first: Move both files to ~/.gem if you don’t already have a key and certificate in that directory. Ensure the file permissions make the key unreadable by others (by default the file is saved securely).
Keep your private key hidden; if it’s compromised, someone can sign packages as you (note: PKI has ways of mitigating the risk of stolen keys; more on that later).
In RubyGems 2 and newer there is no extra work to sign a gem. RubyGems will automatically find your key and certificate in your home directory and use them to sign newly packaged gems.
If your certificate is not self-signed (signed by a third party) RubyGems will attempt to load the certificate chain from the trusted certificates. Use gem cert --add signing_cert.pem
to add your signers as trusted certificates. See below for further information on certificate chains.
If you build your gem it will automatically be signed. If you peek inside your gem file, you’ll see a couple of new files have been added:
$ tar tf your-gem-1.0.gem metadata.gz metadata.gz.sig # metadata signature data.tar.gz data.tar.gz.sig # data signature checksums.yaml.gz checksums.yaml.gz.sig # checksums signature
If you wish to store your key in a separate secure location you’ll need to set your gems up for signing by hand. To do this, set the signing_key
and cert_chain
in the gemspec before packaging your gem:
s.signing_key = '/secure/path/to/gem-private_key.pem' s.cert_chain = %w[/secure/path/to/gem-public_cert.pem]
When you package your gem with these options set RubyGems will automatically load your key and certificate from the secure paths.
Now let’s verify the signature. Go ahead and install the gem, but add the following options: -P HighSecurity
, like this:
# install the gem with using the security policy "HighSecurity" $ sudo gem install your.gem -P HighSecurity
The -P
option sets your security policy – we’ll talk about that in just a minute. Eh, what’s this?
$ gem install -P HighSecurity your-gem-1.0.gem ERROR: While executing gem ... (Gem::Security::Exception) root cert /CN=you/DC=example is not trusted
The culprit here is the security policy. RubyGems has several different security policies. Let’s take a short break and go over the security policies. Here’s a list of the available security policies, and a brief description of each one:
NoSecurity
- Well, no security at all. Signed packages are treated like unsigned packages.
LowSecurity
- Pretty much no security. If a package is signed then RubyGems will make sure the signature matches the signing certificate, and that the signing certificate hasn’t expired, but that’s it. A malicious user could easily circumvent this kind of security.
MediumSecurity
- Better than LowSecurity
and NoSecurity
, but still fallible. Package
contents are verified against the signing certificate, and the signing certificate is checked for validity, and checked against the rest of the certificate chain (if you don’t know what a certificate chain is, stay tuned, we’ll get to that). The biggest improvement over LowSecurity
is that MediumSecurity
won’t install packages that are signed by untrusted sources. Unfortunately, MediumSecurity
still isn’t totally secure – a malicious user can still unpack the gem, strip the signatures, and distribute the gem unsigned.
HighSecurity
- Here’s the bugger that got us into this mess. The HighSecurity
policy is identical to the MediumSecurity
policy, except that it does not allow unsigned gems. A malicious user doesn’t have a whole lot of options here; they can’t modify the package contents without invalidating the signature, and they can’t modify or remove signature or the signing certificate chain, or RubyGems will simply refuse to install the package. Oh well, maybe they’ll have better luck causing problems for CPAN users instead :).
The reason RubyGems refused to install your shiny new signed gem was because it was from an untrusted source. Well, your code is infallible (naturally), so you need to add yourself as a trusted source:
# add trusted certificate gem cert --add ~/.gem/gem-public_cert.pem
You’ve now added your public certificate as a trusted source. Now you can install packages signed by your private key without any hassle. Let’s try the install command above again:
# install the gem with using the HighSecurity policy (and this time # without any shenanigans) $ gem install -P HighSecurity your-gem-1.0.gem Successfully installed your-gem-1.0 1 gem installed
This time RubyGems will accept your signed package and begin installing.
While you’re waiting for RubyGems to work it’s magic, have a look at some of the other security commands by running gem help cert
:
Options: -a, --add CERT Add a trusted certificate. -l, --list [FILTER] List trusted certificates where the subject contains FILTER -r, --remove FILTER Remove trusted certificates where the subject contains FILTER -b, --build EMAIL_ADDR Build private key and self-signed certificate for EMAIL_ADDR -C, --certificate CERT Signing certificate for --sign -K, --private-key KEY Key for --sign or --build -A, --key-algorithm ALGORITHM Select key algorithm for --build from RSA, DSA, or EC. Defaults to RSA. -s, --sign CERT Signs CERT with the key from -K and the certificate from -C -d, --days NUMBER_OF_DAYS Days before the certificate expires -R, --re-sign Re-signs the certificate from -C with the key from -K
We’ve already covered the --build
option, and the --add
, --list
, and --remove
commands seem fairly straightforward; they allow you to add, list, and remove the certificates in your trusted certificate list. But what’s with this --sign
option?
To answer that question, let’s take a look at “certificate chains”, a concept I mentioned earlier. There are a couple of problems with self-signed certificates: first of all, self-signed certificates don’t offer a whole lot of security. Sure, the certificate says Yukihiro Matsumoto, but how do I know it was actually generated and signed by matz himself unless he gave me the certificate in person?
The second problem is scalability. Sure, if there are 50 gem authors, then I have 50 trusted certificates, no problem. What if there are 500 gem authors? 1000? Having to constantly add new trusted certificates is a pain, and it actually makes the trust system less secure by encouraging RubyGems users to blindly trust new certificates.
Here’s where certificate chains come in. A certificate chain establishes an arbitrarily long chain of trust between an issuing certificate and a child certificate. So instead of trusting certificates on a per-developer basis, we use the PKI concept of certificate chains to build a logical hierarchy of trust. Here’s a hypothetical example of a trust hierarchy based (roughly) on geography:
-------------------------- | rubygems@rubygems.org | -------------------------- | ----------------------------------- | | ---------------------------- ----------------------------- | seattlerb@seattlerb.org | | dcrubyists@richkilmer.com | ---------------------------- ----------------------------- | | | | --------------- ---------------- ----------- -------------- | drbrain | | zenspider | | pabs@dc | | tomcope@dc | --------------- ---------------- ----------- --------------
Now, rather than having 4 trusted certificates (one for drbrain, zenspider, pabs@dc, and tomecope@dc), a user could actually get by with one certificate, the “rubygems@rubygems.org” certificate.
Here’s how it works:
I install “rdoc-3.12.gem”, a package signed by “drbrain”. I’ve never heard of “drbrain”, but his certificate has a valid signature from the “seattle.rb@seattlerb.org” certificate, which in turn has a valid signature from the “rubygems@rubygems.org” certificate. Voila! At this point, it’s much more reasonable for me to trust a package signed by “drbrain”, because I can establish a chain to “rubygems@rubygems.org”, which I do trust.
The --sign
option allows all this to happen. A developer creates their build certificate with the --build
option, then has their certificate signed by taking it with them to their next regional Ruby meetup (in our hypothetical example), and it’s signed there by the person holding the regional RubyGems signing certificate, which is signed at the next RubyConf by the holder of the top-level RubyGems certificate. At each point the issuer runs the same command:
# sign a certificate with the specified key and certificate # (note that this modifies client_cert.pem!) $ gem cert -K /mnt/floppy/issuer-priv_key.pem -C issuer-pub_cert.pem --sign client_cert.pem
Then the holder of issued certificate (in this case, your buddy “drbrain”), can start using this signed certificate to sign RubyGems. By the way, in order to let everyone else know about his new fancy signed certificate, “drbrain” would save his newly signed certificate as ~/.gem/gem-public_cert.pem
Obviously this RubyGems trust infrastructure doesn’t exist yet. Also, in the “real world”, issuers actually generate the child certificate from a certificate request, rather than sign an existing certificate. And our hypothetical infrastructure is missing a certificate revocation system. These are that can be fixed in the future…
At this point you should know how to do all of these new and interesting things:
build a gem signing key and certificate
adjust your security policy
modify your trusted certificate list
sign a certificate
In case you don’t trust RubyGems you can verify gem signatures manually:
Fetch and unpack the gem
gem fetch some_signed_gem tar -xf some_signed_gem-1.0.gem
Grab the public key from the gemspec
gem spec some_signed_gem-1.0.gem cert_chain | \ ruby -rpsych -e 'puts Psych.load($stdin)' > public_key.crt
Generate a SHA1 hash of the data.tar.gz
openssl dgst -sha1 < data.tar.gz > my.hash
Verify the signature
openssl rsautl -verify -inkey public_key.crt -certin \ -in data.tar.gz.sig > verified.hash
Compare your hash to the verified hash
diff -s verified.hash my.hash
Repeat 5 and 6 with metadata.gz
OpenSSL
Reference The .pem files generated by –build and –sign are PEM files. Here’s a couple of useful OpenSSL
commands for manipulating them:
# convert a PEM format X509 certificate into DER format: # (note: Windows .cer files are X509 certificates in DER format) $ openssl x509 -in input.pem -outform der -out output.der # print out the certificate in a human-readable format: $ openssl x509 -in input.pem -noout -text
And you can do the same thing with the private key file as well:
# convert a PEM format RSA key into DER format: $ openssl rsa -in input_key.pem -outform der -out output_key.der # print out the key in a human readable format: $ openssl rsa -in input_key.pem -noout -text
There’s no way to define a system-wide trust list.
custom security policies (from a YAML
file, etc)
Simple method to generate a signed certificate request
Support for OCSP, SCVP, CRLs, or some other form of cert status check (list is in order of preference)
Support for encrypted private keys
Some sort of semi-formal trust hierarchy (see long-winded explanation above)
Path discovery (for gem certificate chains that don’t have a self-signed root) – by the way, since we don’t have this, THE ROOT OF THE CERTIFICATE CHAIN MUST BE SELF SIGNED if Policy#verify_root
is true (and it is for the MediumSecurity
and HighSecurity
policies)
Better explanation of X509 naming (ie, we don’t have to use email addresses)
Honor AIA field (see note about OCSP above)
Honor extension restrictions
Might be better to store the certificate chain as a PKCS#7 or PKCS#12 file, instead of an array embedded in the metadata.
Paul Duncan <pabs@pablotron.org> pablotron.org/
This module is used for safely loading Marshal
specs from a gem. The ‘safe_load` method defined on this module is specifically designed for loading Gem specifications.
This module allows for introspection of YJIT, CRuby’s just-in-time compiler. Everything in the module is highly implementation specific and the API might be less stable compared to the standard library.
This module may not exist if YJIT does not support the particular platform for which CRuby is built.