Ruby 3 major updates

The number 3 is very significant in the Ruby 3 release. Be it release version number, making performance 3x faster, or the trio of core contributors(Matz, TenderLove, Koichi). Similarly, there were 3 major goals of Ruby 3: being faster, having better concurrency, and ensuring correctness.

1. Ruby 3 performance

One of the major focuses for Ruby 3 was the performance. In fact, the initial discussion for Ruby 3 was started around it. Matz had set a very ambitious goal of making Ruby 3 times faster.

What is Ruby 3x3?

Before discussing this, let's revisit Ruby's core philosophy.

"I hope to see Ruby help every programmer in the world to be productive, and to enjoy programming, and to be happy." - Matz

About Ruby 3x3, some asked whether the goal was to make Ruby the fastest language? The answer is no. The main goal of Ruby 3x3 was to make Ruby 3 times faster than Ruby 2.

“No language is fast enough.” - Matz

Ruby was not designed to be fastest and if it would have been the goal, Ruby wouldn't be the same as it is today. As Ruby language gets performance boost, it definitely helps our application be faster and scalable.

"In the design of the Ruby language we have been primarily focused on productivity and the joy of programming. As a result, Ruby was too slow." - Matz

There are two areas where performance can be measured: memory and CPU.

CPU optimization

Some enhancements in Ruby internals have been made to improve the speed. The Ruby team has optimized the JIT(Just In Time) compiler from previous versions. Ruby MJIT compiler was first introduced in Ruby 2.6. Ruby 3 MJIT comes with better security and seems to improve web apps’ performance to a greater extent.

MJIT implementation is different from the usual JIT. When methods get called repeatedly e.g. 10000 times, MJIT will pick such methods which can be compiled into native code and put them into a queue. Later MJIT will fetch the queue and convert them to native code.

Please check JIT vs MJIT for more details.

Memory optimization

Ruby 3 comes with an enhanced garbage collector. It has python's buffer-like API which helps in better memory utilization. Since Ruby 1.8, Ruby has continuously evolved in Garbage collection algorithms.

Automatic Garbage Compaction

The latest change in garbage collection is Garbage Compaction. It was introduced in Ruby 2.7 where the process was a bit manual. But in version 3 it is fully automatic. The compactor is invoked aptly to make sure proper memory utilization.

Objects Grouping

The garbage compactor moves objects in the heap. It groups dispersed objects together at a single place in the memory so that this memory can be used even by heavier objects later.

2. Parallelism and Concurrency in Ruby 3

Concurrency is one of the important aspects of any programming language. Matz feels that Threads are not the right level of abstraction for Ruby programmers to use.

“I regret adding Threads.” - Matz

Ruby 3 makes it a lot easier to make applications where concurrency is a major focus. There are several features and improvements added in Ruby 3 related to concurrency.

Fibers

Fibers are considered a disruptive addition in Ruby 3. Fibers are light-weight workers which appear like Threads but have some advantages. It consumes less memory than Threads. It gives greater control to the programmer to define code segments that can be paused or resumed resulting in better I/O handling.

Falcon Rack web server uses Async Fibers internally. This allows Falcon to not block on I/O. Asynchronously managing I/O gives a great uplift to the Falcon server to serve requests concurrently.

Fiber Scheduler

Fiber Scheduler is an experimental feature added in Ruby 3. It was introduced to intercept blocking operations such as I/O. The best thing is that it allows lightweight concurrency and can easily integrate into the existing codebase without changing the original logic. It's an interface and that can be implemented by creating a wrapper for a gem like EventMachine or Async. This interface design allows separation of concerns between the event loop implementation and the application code.

Following is an example to send multiple HTTP requests concurrently using Async.

require 'async'
require 'net/http'
require 'uri'

LINKS = [
  'https://bigbinary.com',
  'https://basecamp.com'
]

Async do
  LINKS.each do |link|
    Async do
      Net::HTTP.get(URI(link))
    end
  end
end

Please check fibers for more details.

Ractors (Guilds)

As we know Ruby’s global VM lock (GVL) prevents most Ruby Threads from computing in parallel. Ractors work around the GVL to offer better parallelism. Ractor is an Actor-Model like concurrent abstraction designed to provide a parallel execution without thread-safety concerns.

Ractors allows Threads in different Ractors to compute at the same time. Each Ractor has at least one thread, which may contain multiple fibers. In a Ractor, only a single thread is allowed to execute at a given time.

The following program returns the square root of a really large number. It calculates the result for both numbers in parallel.

# Math.sqrt(number) in ractor1, ractor2 run in parallel

ractor1, ractor2 = *(1..2).map do
  Ractor.new do
    number = Ractor.recv
    Math.sqrt(number)
  end
end

# send parameters
ractor1.send 3**71
ractor2.send 4**51

p ractor1.take #=> 8.665717809264115e+16
p ractor2.take #=> 2.251799813685248e+15

3. Static Analysis

We need tests to ensure correctness of our program. However by its very nature tests could mean code duplication.

“I hate tests because they aren't DRY.” - Matz

To ensure the correctness of a program, static analysis can be a great tool in addition to tests.

The static analysis relies on inline type annotations which aren't DRY. The solution to address this challenge is having .rbs files parallel to our .rb files.

RBS

RBS is a language to describe the structure of a Ruby program. It provides us an overview of the program and how overall classes, methods, etc. are defined. Using RBS, we can write the definition of Ruby classes, modules, methods, instance variables, variable types, and inheritance. It supports commonly used patterns in Ruby code, and advanced types like unions and duck typing.

The .rbs files are something similar to .d.ts files in TypeScript. Following is a small example of how a .rbs file looks like. The advantage of having a type definition is that it can be validated against both implementation and execution.

The below example is pretty self-explanatory. One thing we can note here though, each_post accepts a block or returns an enumerator.

# user.rbs

class User
  attr_reader name: String
  attr_reader email: String
  attr_reader age: Integer
  attr_reader posts: Array[Post]

  def initialize: (name: String,
                   email: String,
                   age: Integer) -> void

  def each_post: () { (Post) -> void } -> void
                   | () -> Enumerator[Post, void]
end

Please check RBS gem documentation for more details.

Typeprof

Introducing type definition was a challenge because there is already a lot of existing Ruby code around and we need a tool that could automatically generate the type signature. Typeprof is a type analysis tool that reads plain Ruby code and generates a prototype of type signature in RBS format by analyzing the methods, and its usage. Typeprof is an experimental feature. Right now only small subset of ruby is supported.

“Ruby is simple in appearance, but is very complex inside, just like our human body.” - Matz

Let's see an example.

# user.rb
class User

  def initialize(name:, email:, age:)
    @name, @email, @age = name, email, age
  end

  attr_reader :name, :email, :age
end

User.new(name: "John Doe", email: 'john@example.com', age: 23)

Output

$ typeprof user.rb

# Classes
class User

  attr_reader name : String
  attr_reader email : String
  attr_reader age : Integer

  def initialize : (name: String,
                    email: String,
                    age: Integer) -> [String, String, Integer]

end

Other Ruby 3 features and changes

In the 7-year period, the Ruby community has seen significant improvement in performance and other aspects. Apart from major goals, Ruby 3 is an exciting update with lots of new features, handy syntactic changes, and new enhancements. In this section, we will discuss some notable features.

“We are making Ruby even better.” - Matz

One-line pattern matching syntax change

Previously one-line pattern matching used the keyword in. Now it's replaced with =>.

Ruby 2.7

  { name: 'John', role: 'CTO' } in {name:}
  p name # => 'John'

Ruby 3.0

  { name: 'John', role: 'CTO' } => {name:}
  p name # => 'John'

Find pattern

The find pattern was introduced in Ruby 2.7 as an experimental feature. This is now part of Ruby 3.0. It is similar to pattern matching in Elixir or Haskell.

users = [
  { name: 'Oliver', role: 'CTO' },
  { name: 'Sam', role: 'Manager' },
  { role: 'customer' },
  { name: 'Eve', city: 'New York' },
  { name: 'Peter' },
  { city: 'Chicago' }
]

users.each do |person|
  case person
  in { name:, role: 'CTO' }
    p "#{name} is the Founder."
  in { name:, role: designation }
    p "#{name} is a #{designation}."
  in { name:, city: 'New York' }
    p "#{name} lives in New York."
  in {role: designation}
    p "Unknown is a #{designation}."
  in { name: }
    p "#{name}'s designation is unknown."
  else
    p "Pattern not found."
  end
end

"Oliver is the Founder."
"Sam is a Manager."
"Unknown is a customer."
"Eve lives in New York."
"Peter's designation is unknown."
"Pattern not found."

Endless Method definition

This is another syntax enhancement that is optional to use. It enables us to create method definitions without end keyword.

def: increment(x) = x + 1

p increment(42) #=> 43

Except method in Hash

Sometimes while working on a non Rails app I get undefined method except. The except method was available only in Rails. In Ruby 3 Hash#except was added to Ruby itself.

user = { name: 'Oliver', age: 29, role: 'CTO' }

user.except(:role) #=> {:name=> "Oliver", :age=> 29}

Memory View

This is again an experimental feature. This is a C-API that will allow extension libraries to exchange raw memory area. Extension libraries can also share metadata of memory area that consists of shape and element format. It was inspired by Python’s buffer protocol.

Arguments forwarding

Arguments forwarding (...) now supports leading arguments.

It is helpful in method_missing, where we need method name as well.

def method_missing(name, ...)
  if name.to_s.end_with?('?')
    self[name]
  else
    fallback(name, ...)
  end
end

Other Notable changes

• Pasting in IRB is much faster.
• The order of backtrace had been reversed. The error message and line number are printed first, rest of the backtrace is printed later.
• Hash#transform_keys accepts a hash that maps old keys with new keys.
• Interpolated String literals are no longer frozen when # frozen-string-literal: true is used.
• Symbol#to_proc now returns a lambda Proc.
• Symbol#name has been added, which returns the symbol's name as a frozen string.

Many other changes can be checked at Ruby 3 News for more details.

Transition

A lot of core libraries have been modified to fit the Ruby 3 goal needs. But this doesn't mean that our old applications will suddenly stop working. The Ruby team has made sure that these changes are backward compatible. We might see some deprecation warnings in our existing code. The developers can fix these warnings to smoothly transition from an old version to the new version. We are all set to use new features and get all new performance improvements.

Conclusion

With great improvements in performance, memory utilization, static analysis, and new features like Ractors and schedulers, we have great confidence in the future of Ruby. With Ruby 3, the applications can be more scalable and more enjoyable to work with. The coming 2021 is not just a new year but rather a new era for all Rubyists. We at BigBinary thank everyone who contributed towards the Ruby 3 release directly or indirectly.

Happy Holidays and Happy New Year folks!!!

Usage before Ruby 3

The following example shows how we used to apply transform_keys:

# 1. Declare address hash
irb(main)> address = {House: 'Kizhakkethara', house_no: 123, locality: 'India'}
=> {:House=>"Kizhakkethara", :house_no=>123, :locality=>"India"}

# 2. Lowercase all the keys
irb(main)> address.transform_keys(&:downcase)
=> {:house=>"Kizhakkethara", :house_no=>123, :locality=>"India"}

# 3. Replace a particular key with a new key along with lowercasing
irb(main)* address.transform_keys do |key|
irb(main)*   new_key = key
irb(main)*   if key == :locality
irb(main)*     new_key = :country
irb(main)*   end
irb(main)*   new_key.to_s.downcase.to_sym
irb(main)> end
=> {:house=>"Kizhakkethara", :house_no=>123, :country=>"India"}

Although the changes required are trivial, we ended up writing a block to do the job. But what happens when the number of keys that needs to be transformed increases? Do we need to write n-number of conditions within a block? Not anymore!

Introducing Hash#transform_keys with hash argument

Let's take the same example and provide a hash, which will be used for the transformation:

# 1. Declare address hash
irb(main)> address = {House: 'Kizhakkethara', house_no: 123, locality: 'India'}
=> {:House=>"Kizhakkethara", :house_no=>123, :locality=>"India"}

# 2. Provide hash with transform_keys
irb(main)> address.transform_keys({House: :house, locality: :country})
=> {:house=>"Kizhakkethara", :house_no=>123, :country=>"India"}

That does the job. But let's try to improve this code. Ultimately what happens when we invoke that method is that it goes through each of the keys in our variable and maps the existing keys to the new keys. The transform_keys method accepts a block as a parameter. Thus let's pass in the downcase method as a Proc argument:

# 1. Passing in block parameters
irb(main)> address.transform_keys({locality: :country}, &:downcase)
=> {:house=>"Kizhakkethara", :house_no=>123, :country=>"India"}

An important point to be noted about the block parameter is that, it's only applied to keys which are not specified in the hash argument.

Other common use cases

Transforming params received in the Rails controller

# 1. Declare params
irb(rails)> params = ActionController::Parameters.new({"firstName"=>"oliver", "lastName"=>"smith", "email"=>"oliver@bigbinary.com"})
=> <ActionController::Parameters {"firstName"=>"oliver", "lastName"=>"smith", "email"=>"oliver@bigbinary.com"} permitted: false>

# 2. Convert camelCase to snake_case using block parameter
irb(rails)> params.permit(:firstName, :lastName, :email).transform_keys(&:underscore)
=> <ActionController::Parameters {"first_name"=>"oliver", "last_name"=>"smith", "email"=>"oliver@bigbinary.com"} permitted: true>

# 3. Or using hash argument
irb(rails)> params.permit(:firstName, :lastName, :email).transform_keys({firstName: 'first_name', lastName: 'last_name'})
=> <ActionController::Parameters {"first_name"=>"oliver", "last_name"=>"smith", "email"=>"oliver@bigbinary.com"} permitted: true>

Slicing hash along with key transformation

irb(main)> address.transform_keys({locality: :country}).slice(:house_no, :country)
=> {:house_no=>123, :country=>"India"}

Transforming keys in place using bang counterpart

irb(main)> address.transform_keys!({locality: :country}, &:downcase)
irb(main)> address
=> {:house=>"Kizhakkethara", :house_no=>123, :country=>"India"}

References

• Discussions regarding this feature can be found here.
• Commit for this feature can be found here.

Why do we need Hash#except?

At times, we need to print or log everything except some sensitive data. Let's say we want to print user details in the logs but not passwords.

Before Ruby 3 we could have achieved it in the following ways:

irb(main):001:0> user_details = { name: 'Akhil', age: 25, address: 'India', password: 'T:%g6R' }

# 1. Reject with a block and include?
irb(main):003:0> puts user_details.reject { |key, _| key == :password }
=> { name: 'Akhil', age: 25, address: 'India' }

# 2. Clone the hash with dup, tap into it and delete that key/value from the clone
irb(main):005:0> puts user_details.dup.tap { |hash| hash.delete(:password) }
=> { name: 'Akhil', age: 25, address: 'India' }

We know that ActiveSupport already comes with Hash#except but for a simple Ruby application using ActiveSupport would be overkill.

Ruby 3

To make the above task easier and more explicit, Ruby 3 adds Hash#except to return a hash excluding the given keys and their values:

irb(main):001:0> user_details = { name: 'Akhil', age: 25, address: 'India', password: 'T:%g6R' }
irb(main):002:0> puts user_details.except(:password)
=> { name: 'Akhil', age: 25, address: 'India' }

irb(main):003:0> db_info = YAML.safe_load(File.read('./database.yml'))
irb(main):004:0> puts db_info.except(:username, :password)
=> { port: 5432, database_name: 'example_db_production' }

Check out the commit for more details. Discussion around it can be found here.

One of the important aspects of Ruby 3.0 is optimization. The part of that optimization is the introduction of name method for Symbol. In this blog, we will take a look at what name method of class Symbol does and why it was introduced. The new name method is introduced on Symbol to simply convert a symbol into a string. Symbol#name returns a string. Let's see how it works.

irb(main):001:0> :simba.name
=> 'simba'
irb(main):002:0> :simba.name.class
=> String
irb(main):003:0> :simba.name === :simba.name
=> true

Wait what? Don't we have to_s to convert a symbol into a string. Most of us have used to_s method on a Symbol. The to_s method returns a String object and we can simply use it. But why name?

Using to_s is okay in most cases. But the problem with to_s is that it creates a new String object every time we call it on a symbol. We can verify this in irb.

irb(main):023:0> :simba.to_s.object_id
=> 260
irb(main):024:0> :simba.to_s.object_id
=> 280

Creating a new object for every symbol to a string conversion allocates new memory which increases overhead. The light was thrown on this issue by schneems (Richard Schneeman) in a talk at RubyConf Thailand where he showed how Symbol#to_s allocation causes significant overhead in ActiveRecord. This inspired Ruby community to have a new method name on Symbol which returns a frozen string object. This reduces the string allocations dramatically which results in reducing overhead.

irb(main):001:0> :simba.name.frozen?
=> true
irb(main):002:0> :simba.name.object_id
=> 200
irb(main):003:0> :simba.name.object_id
=> 200

The reason to bring this feature was that most of the times we want a simple string representation for displaying purpose or to interpolate into another string. The result of to_s is rarely mutated directly. By introducing this method we save a lot of objects which helps in optimization. Now we know the benefits of name, we should prefer using name over to_s when we don't want to mutate a string.

For more information on discussion, official documentation, please head on to Feature #16150 discussion, Pull request and Ruby 3.0 official release preview.

# endless method definition
>> def raise_to_power(number, power) = number ** power

>> raise_to_power(2, 5)

=> 32

The discussion around it can be found here. Check out the pull request for more details on this.