to navigateEnterto select Escto close

    Handling idempotency when sending emails using Sidekiq

    Now that we have covered unit testing and background jobs with Sidekiq, let's add a new feature to send reminder emails to users with outstanding tasks.

    In this chapter we will discuss concepts like idempotency, mail delivery windows, setting user preferences, and more.


    These are the basic requirements of the feature :

    • Send reminder emails to users with pending tasks.
    • Users should have an option to enable/disable the email notifications.
    • User can set preferred hour i.e delivery time to receive mail with all pending tasks, if any.
    • Users should not be able to update the email delivery hour unless the email notifications are enabled.

    Mail delivery feature

    Technical design

    To implement this feature, we need to introduce the following changes:

    On the backend

    • For scheduling periodic Sidekiq jobs, we can use the sidekiq-cron gem, which helps us to schedule the worker, which will be sending the mail.

    • In the development environment, instead of sending an email to the user's email address, we can preview the email in our browser using the letter_opener gem.

    • Mail will be sent to each user at 10 AM by default, or at their preferred time, which we will store in a new table called preference. This preferences table consists of notification_delivery_hour,receive_email and user columns.

    • Create user_notifications table with last_notification_sent_date and user columns.

    • In user_notification model, add validations to ensure that the last_notification_sent_date attribute is present, has a valid date format, and the date is not in the past.

    • We will be creating a default preference for the user whenever a new user is created.

    • We will be saving the default preference value in a constant DEFAULT_NOTIFICATION_DELIVERY_HOUR, which will be loaded automatically from config/initializers/constants.rb, by Rails, after it loads the framework plus any gems and plugins in our application.

    • Add PreferencesController which will add the ability for the user to update their preference for if and when to receive the email.

      Inside the PreferencesController, we will add a custom action called mail which will handle enabling and disabling the email notification.

    • Mail sending can often fail due to network errors, among others. When that happens, Sidekiq will retry based on rules we set in the initializer.

    On the frontend

    • Define preference related axios connectors.

    • We will add a MyPreferences component which will handle the preferences of the user.

    • Delegate the job of taking in the preferred time for mail delivery to a PreferenceForm component within the MyPreferences component.

    • Add a function fetchPreferenceDetails in the MyPreferences container component which will fetch preference details like notification_delivery_hour and receive_email.

    • Add a function updateEmailNotification in the MyPreferences component which will handle enabling/disabling email notifications.

    • Similarly, add a function updatePreference in the MyPreferences container component which will update notification_delivery_hour.

    Sidekiq worker

    To send each batch of reminder emails, we will schedule a TodoNotificationsWorker to run every hour and delegate the job of finding the recipients to a TodoNotificationsService.

    This service will ultimately invoke separate UserNotificationWorker's for each recipient it finds.

    Invoking separate workers helps to avoid having a single point of failure in the batch, which may bring the whole delivery system down.

    We will also create a scheduler file to store cron values that will help schedule our TodoNotificationsWorker at the desired time interval for each environment.

    The parent Sidekiq worker will then invoke TodoNotificationsService, which will find all the users with pending tasks but no entries in a new table called user_notifications.

    What is idempotency?

    We need our service to be idempotent, ie: duplicate mail shouldn't be sent every time we retry.

    From a RESTful service standpoint, an operation or service call is idempotent when clients can perform a duplicate request harmlessly. In other words, making multiple identical requests has the same effect as making a single request.

    Note that while idempotent operations produce the same result on the server, the response itself may not be the same; e.g. a resource's state may change between requests.

    To ensure this, we will use the user_notifications table from earlier to associate each user with a last_notification_sent_date.

    If a user has an entry for today in user_notifications, it means there's no need to try again.

    Action Mailer

    Action Mailer allows you to send emails from your application using mailer classes and views. They inherit from ActionMailer::Base and live in app/mailers.

    To make sure that the email has truly been sent to the user, we will make use of Action Mailer's after_action callback to safely set the last_notification_sent_date value only after the mail has been sent successfully.

    Mailers have methods called "actions" and they use views to structure their content.

    A controller generates content like say HTML, which will be sent back to the client, whereas a Mailer creates a message to be delivered via email.

    We will be creating the TaskMailer mailer and task_mailer view.

    We are now ready to start coding. Let us dive in.

    Setting up base

    Open the Gemfile.rb and add the following lines:

    1# For periodic sidekiq jobs
    2gem "sidekiq-cron"
    4# For opening mails in development env
    5gem "letter_opener", :group => :development

    Install the gems:

    1bundle install

    In order for the letter_opener gem to open the mails automatically in our browser, we need to set the following in config/environments/development.rb:

    1config.action_mailer.delivery_method = :letter_opener
    2config.action_mailer.perform_deliveries = true

    Now any email delivery will pop up in our browser instead of being sent via the internet. The messages will be stored in ./tmp/letter_opener.

    Creating models

    1bundle exec rails g model user_notification
    2bundle exec rails g model preference

    Go to db/migrate folder and paste the following lines of code for the CreateUserNotifications migration:

    1class CreateUserNotifications < ActiveRecord::Migration[6.1]
    2  def change
    3    create_table :user_notifications do |t|
    4 :last_notification_sent_date, null: false
    5      t.references :user, foreign_key: true
    6      t.index [:user_id, :last_notification_sent_date],
    7        name: :index_user_preferences_on_user_id_and_notification_sent_date,
    8        unique: true
    9      t.timestamps
    10    end
    11  end

    Also, paste the following lines of code for the CreatePreferences migration:

    1class CreatePreferences < ActiveRecord::Migration[6.1]
    2  def change
    3    create_table :preferences do |t|
    4      t.integer :notification_delivery_hour
    5      t.boolean :receive_email, default: true, null: false
    6      t.references :user, foreign_key: true
    7      t.timestamps
    8    end
    9  end

    Run the migrations:

    1bundle exec rails db:migrate

    Adding validations

    Now let's setup UserNotification model and add validations for last_notification_sent_date.

    Open app/models/user_notification.rb and add the following lines to it:

    1class UserNotification < ApplicationRecord
    2  belongs_to :user
    4  validates :last_notification_sent_date, presence: true
    5  validate :last_notification_sent_date_is_valid_date
    6  validate :last_notification_sent_date_cannot_be_in_the_past
    8  private
    10  def last_notification_sent_date_is_valid_date
    11    if last_notification_sent_date.present?
    12      Date.parse(last_notification_sent_date.to_s)
    13    end
    14  rescue ArgumentError
    15    errors.add(:last_notification_sent_date, "must be a valid date")
    16  end
    18  def last_notification_sent_date_cannot_be_in_the_past
    19    if last_notification_sent_date.present? &&
    20        last_notification_sent_date <
    21      errors.add(:last_notification_sent_date, "can't be in the past")
    22    end
    23  end

    Now let's setup our Preference model along with validations in app/models/preference.rb:

    1class Preference < ApplicationRecord
    2  belongs_to :user
    4  validates :notification_delivery_hour, presence: true,
    5    numericality: { only_integer: true },
    6    inclusion: {
    7      in: 0..23,
    8      message: "value should be between 0 and 23"
    9    }

    Currently, we are only providing the ability to set the preference for when to receive an email notification.

    Thus in the above code, we have added the validations for the notification_delivery_hour attribute which makes sure that it's present, and it only accepts integers ranging from 0-23.

    Add the following line in config/initializers/constants.rb file::

    1module Constants
    2  is_sqlite_db = ActiveRecord::Base.connection_db_config.configuration_hash[:adapter] == "sqlite3"
    3  DB_REGEX_OPERATOR = is_sqlite_db ? "REGEXP" : "~*"

    We are setting the DEFAULT_NOTIFICATION_DELIVERY_HOUR in our constants.rb file instead of a specific model. This is because the constant is not limited to a specific context and will be used across different files. Thus it makes sense to add it to a global Constants context instead.

    Now let's set the associations correctly in our User model too:

    1class User < ApplicationRecord
    2  # previous code if any
    3  has_many :user_notifications, dependent: :destroy, foreign_key: :user_id
    4  has_one  :preference, dependent: :destroy, foreign_key: :user_id
    6  # previous code if any
    7  before_save :to_lowercase
    8  before_create :build_default_preference
    10  private
    12    # previous code if any
    14    def to_lowercase
    15      email.downcase!
    16    end
    18    def build_default_preference
    19      self.build_preference(notification_delivery_hour: Constants::DEFAULT_NOTIFICATION_DELIVERY_HOUR)
    20    end

    The private method, build_default_preference, will create a default preference for the user whenever a new user is created.

    Inside the build_default_preference method we have used the build_preference method. Let's take a look at where it came from.

    When we declare a belongs_to or has_one association, the declaring class automatically will have access to a build_association method related to the association. In this case, because of the has_one :preference association in User model, Rails adds the build_preference method to the User class.

    In case of has_many or has_and_belongs_to_many associations, we should use the method instead of the build_association method. For example, if we were to declare a has_many :preferences association in the User model, we would have to use the method given that Rails won't add the build_preferences method for an association that is not has_one.

    Adding preferences for existing users

    The build_preference method we had added in the previous section will only add a default preference for new users. The preference for all existing users will remain nil. There are two ways by which we can fix this issue:

    1. Deleting all users from the console and creating new users. Newly added users will have a default preference but we shouldn't take this approach because:

      • It's considered a bad practice.

      • Once the application is deployed, we shouldn't delete users from console in production environment.

    2. A better way to do this would be to add a migration which will add default preference for all existing users where preference is nil.

    If you recall, we have done something similar in the Adding slug to task chapter where we had used a migration to add slugs to existing task records.

    Before moving on towards adding a migration to fix the issue, you should attempt to do this by yourself. If you need help you can refer to the Adding slug to task chapter.

    Now, let's us add a migration to fix this issue. Run the following command to generate a migration:

    1bundle exec rails g migration AddDefaultPreferenceToExistingUsers

    The above command with generate a migration file. Inside the migration file add the following lines of code:

    1class AddDefaultPreferenceToExistingUsers < ActiveRecord::Migration[6.1]
    2 def up
    3    users_with_nil_preference = User.where.missing(:preference)
    4    users_with_nil_preference.each do |user|
    5      user.send(:build_default_preference)
    7    end
    8  end
    10  def down
    11    User.find_each do |user|
    12      user.preference.delete
    13    end
    14  end

    We have used the missing method which performs a left outer join operation on parent and child tables then uses a where clause to identify the missing relations. In this case we are querying all user records which have a missing preference relation and then adding a default preference for all those user records.

    You can read more about the missing method from official documentation.

    Run the following command to execute the migration:

    1bundle exec rails db:migrate

    Creating the controller

    Create a new controller for the preference model by running the following command:

    1bundle exec rails generate controller Preferences

    Open app/controllers/preferences_controller.rb and add the following lines to it:

    1class PreferencesController < ApplicationController
    2  before_action :authenticate_user_using_x_auth_token
    3  before_action :load_preference
    5  def show
    6    render status: :ok, json: { preference: @preference }
    7  end
    9  def update
    10    if @preference.update(preference_params)
    11      render status: :ok, json: {
    12        notice: t('successfully_updated', entity: 'Preference')
    13      }
    14    else
    15      error = @preference.errors.full_messages.to_sentence
    16      render status: :unprocessable_entity, json: { error: error }
    17    end
    18  end
    20  def mail
    21    if @preference.update(receive_email: preference_params[:receive_email])
    22      render status: :ok,
    23        json: {
    24          notice: t(
    25            "preference.mail.notification_status",
    26            status: @preference.receive_email ? "enabled" : "disabled"
    27          )
    28        }
    29    else
    30      error = @preference.errors.full_messages.to_sentence
    31      render status: :unprocessable_entity, json: { error: error }
    32    end
    33  end
    35  private
    37    def preference_params
    38      params.require(:preference).permit(:notification_delivery_hour, :receive_email)
    39    end
    41    def load_preference
    42      @preference = current_user.preference
    43      unless @preference
    44        render status: :not_found, json: { error: t("not_found", entity: "Preference") }
    45      end
    46    end

    We have added a mail action in the PreferencesController which will handle the logic to enable and disable the email notifications. Now, the question arises that why we need another action when updating a record should be handled by the update action. We want to do so because the mail action is only concerned with updating the receive_email attribute of a preference record and the response for that will also be different from the update action. Therefore, it is better to add a custom action.

    We have created a load_preference method to load the preference before the show, update and mail actions are called. Since each user has one associated preference record we can query it using current_user.preference inside the load_preference method. Hence appending the preference id to the preference routes or sending it as a request param while making a request is not required in this case.

    Now, let's create the route for the preference. We should declare preference as a singular resource because we do not need a resource identifier in the routes. Open the routes.rb and append the following line:

    1Rails.application.routes.draw do
    2  constraints(lambda { |req| req.format == :json }) do
    3    # ---Previous Routes---
    4    resource :preference, only: %i[show update] do
    5      patch :mail, on: :collection
    6    end
    7  end
    9  root "home#index"
    10  get '*path', to: 'home#index', via: :all

    We have added a collection route called mail for the preference resource. Any patch request on preference/mail will invoke the mail action inside the PreferencesController.

    Note that we have used a patch verb for the mail route over the put verb. This is so because a put verb is used when the entire entity needs to be updated whereas a patch request is used when the entity needs to be updated partially. In our case, we are only updating the receive_email attribute of a preference record thus the use of patch verb.

    Why we do not need a preference policy?

    Policies prevent any unauthorized access of database records. In case of preferences, a user should not be able to access the preferences of another user. We do not need a policy to ensure this because we are using the preferences association of the User model to load the preference inside the PreferencesController.

    Doing so will only load the preference of the current_user thus eliminating the need for authorization using a policy.

    Building the UI for user preference

    Since we have already added APIs for handling user preference, let's now build the views for it.

    Let's create a new file to define all the preference related APIs:

    1touch app/javascript/src/apis/preferences.js

    Open the apis/preferences.js and paste the following lines:

    1import axios from "axios";
    3const show = () => axios.get("/preference");
    5const mail = ({ payload }) => axios.patch(`/preference//mail`, payload);
    7const update = ({ payload }) => axios.put("/preference", payload);
    9const preferencesApi = {
    10  show,
    11  update,
    12  mail,
    15export default preferencesApi;

    Now, let's create the preference component:

    1mkdir -p app/javascript/src/components/MyPreferences
    2touch app/javascript/src/components/MyPreferences/index.jsx

    Paste the following lines into index.jsx:

    1import React, { useState, useEffect } from "react";
    3import preferencesApi from "apis/preferences";
    4import Container from "components/Container";
    5import PageLoader from "components/PageLoader";
    6import { getFromLocalStorage } from "helpers/storage";
    8import PreferenceForm from "./Form";
    10const MyPreferences = () => {
    11  const [notificationDeliveryHour, setNotificationDeliveryHour] = useState("");
    12  const [receiveEmail, setReceiveEmail] = useState(true);
    13  const [loading, setLoading] = useState(false);
    14  const [pageLoading, setPageLoading] = useState(true);
    15  const [preferenceId, setPreferenceId] = useState("");
    16  const userId = getFromLocalStorage("authUserId");
    18  const updatePreference = async () => {
    19    setLoading(true);
    20    try {
    21      await preferencesApi.update({
    22        payload: {
    23          notification_delivery_hour: notificationDeliveryHour,
    24          receive_email: receiveEmail,
    25        },
    26      });
    27    } catch (error) {
    28      logger.error(error);
    29    } finally {
    30      setLoading(false);
    31    }
    32  };
    34  const updateEmailNotification = async emailNotificationStatus => {
    35    setLoading(true);
    36    try {
    37      await preferencesApi.mail({
    38        id: preferenceId,
    39        payload: {
    40          receive_email: emailNotificationStatus,
    41        },
    42      });
    43    } catch (error) {
    44      logger.error(error);
    45    } finally {
    46      setLoading(false);
    47    }
    48  };
    50  const fetchPreferenceDetails = async () => {
    51    try {
    52      const { data } = await;
    53      setNotificationDeliveryHour(data.preference.notification_delivery_hour);
    54      setReceiveEmail(data.preference.receive_email);
    55      setPreferenceId(;
    56    } catch (error) {
    57      logger.error(error);
    58    } finally {
    59      setPageLoading(false);
    60    }
    61  };
    63  useEffect(() => {
    64    fetchPreferenceDetails();
    65  }, []);
    67  if (pageLoading || !userId || !preferenceId) {
    68    return (
    69      <div className="w-screen h-screen">
    70        <PageLoader />
    71      </div>
    72    );
    73  }
    75  return (
    76    <Container>
    77      <PreferenceForm
    78        notificationDeliveryHour={notificationDeliveryHour}
    79        setNotificationDeliveryHour={setNotificationDeliveryHour}
    80        receiveEmail={receiveEmail}
    81        setReceiveEmail={setReceiveEmail}
    82        loading={loading}
    83        updatePreference={updatePreference}
    84        updateEmailNotification={updateEmailNotification}
    85      />
    86    </Container>
    87  );
    90export default MyPreferences;

    Here, fetchPreferenceDetails function uses the show axios API call from preference related axios APIs. Similarly, the updatePreference function will be using the update axios API.

    As you can see we have used a PreferenceForm component which is not yet created. Let's create it:

    1mkdir -p app/javascript/src/components/MyPreferences/Form
    2touch app/javascript/src/components/MyPreferences/Form/index.jsx

    Paste the following lines in the MyPreferences/Form/index.jsx:

    1import React from "react";
    3import classnames from "classnames";
    4import Button from "components/Button";
    5import Input from "components/Input";
    6import Select from "react-select";
    8const defaultTimezone = "UTC";
    10const PreferenceForm = ({
    11  notificationDeliveryHour,
    12  setNotificationDeliveryHour,
    13  receiveEmail,
    14  setReceiveEmail,
    15  loading,
    16  updatePreference,
    17  updateEmailNotification,
    18}) => {
    19  const onHandleDeliveryHourChange = event => {
    20    const regex = /^[0-9\b]*$/;
    21    const deliveryHour =;
    22    if (!regex.test(deliveryHour)) return null;
    24    return setNotificationDeliveryHour(deliveryHour);
    25  };
    27  const handleSubmit = event => {
    28    event.preventDefault();
    29    if (receiveEmail) {
    30      updatePreference();
    31    }
    32  };
    34  const handleEmailNotificationChange = e => {
    35    setReceiveEmail(;
    36    return updateEmailNotification(;
    37  };
    39  return (
    40    <form className="max-w-lg mx-auto" onSubmit={handleSubmit}>
    41      <div className="flex justify-between text-bb-gray-600 mt-10 mb-2">
    42        <h1 className="pb-3 mt-5 text-2xl leading-5 font-bold">
    43          Pending Tasks in Email
    44        </h1>
    45      </div>
    47      <div
    48        className={classnames("flex  items-baseline space-x-1", {
    49          "text-bb-gray-700": receiveEmail,
    50          "text-bb-gray-600": !receiveEmail,
    51        })}
    52      >
    53        <input
    54          type="checkbox"
    55          checked={receiveEmail}
    56          id="receiveEmail"
    57          onChange={handleEmailNotificationChange}
    58        />
    59        <span>
    60          Send me a daily email of the pending tasks assigned to me.
    61          <br /> No email will be sent if there are no pending tasks.
    62        </span>
    63      </div>
    65      <div
    66        className={classnames("flex space-x-4 items-center", {
    67          "text-bb-gray-700": receiveEmail,
    68          "text-bb-gray-600": !receiveEmail,
    69        })}
    70      >
    71        <p className="text-sm font-medium mt-6 leading-5 ">
    72          Delivery Time (Hours)
    73        </p>
    74        <Input
    75          type="number"
    76          placeholder="Enter hour"
    77          disabled={!receiveEmail}
    78          min={0}
    79          max={23}
    80          value={notificationDeliveryHour}
    81          onChange={onHandleDeliveryHourChange}
    82        />
    83        <p className="mt-6 font-extrabold">(UTC)</p>
    84      </div>
    86      <div className="w-2/6">
    87        <Button
    88          type="submit"
    89          buttonText="Schedule Email"
    90          className={classnames({
    91            "cursor-not-allowed bg-opacity-60": !receiveEmail,
    92          })}
    93          loading={loading}
    94        />
    95      </div>
    96    </form>
    97  );
    100export default PreferenceForm;

    The PreferenceForm component contains a checkbox for enabling and disabling the email notifications. When the checkbox value changes, updateEmailNotification function defined inside the MyPreferences component will be called and it will in turn make an API for mail action in the PreferencesController.

    There is a updatePreference function as well which will be called upon submitting the form and it will make an API request to update the logged in user's preference.

    We need to provide an Input component from where the user can set their preferred time for mail delivery.

    We have to constraint the values that this component can accept, when it's type is set to be number.

    Thus let's update our reusable Input component to provide this functionality:

    1import React from "react";
    3import classnames from "classnames";
    4import PropTypes from "prop-types";
    6const Input = ({
    7  type = "text",
    8  label,
    9  value,
    10  onChange,
    11  placeholder,
    12  min,
    13  max,
    14  required = true,
    15}) => {
    16  return (
    17    <div className="mt-6">
    18      {label && (
    19        <label className="block text-sm font-medium leading-5 text-bb-gray-700">
    20          {label}
    21        </label>
    22      )}
    23      <div className="mt-1 rounded-md shadow-sm">
    24        <input
    25          type={type}
    26          required={required}
    27          value={value}
    28          onChange={onChange}
    29          placeholder={placeholder}
    30          min={min}
    31          max={max}
    32          className={classnames(
    33            "block w-full px-3 py-2 placeholder-gray-400 transition duration-150 ease-in-out border border-gray-300 rounded-md appearance-none focus:outline-none focus:shadow-outline-blue focus:border-blue-300 sm:text-sm sm:leading-5",
    34            [className]
    35          )}
    36        />
    37      </div>
    38    </div>
    39  );
    42Input.propTypes = {
    43  type: PropTypes.string,
    44  label: PropTypes.string,
    45  value: PropTypes.node,
    46  placeholder: PropTypes.string,
    47  onChange: PropTypes.func,
    48  required: PropTypes.bool,
    49  min: PropTypes.number,
    50  max: PropTypes.number,
    53export default Input;

    Finally, our Preference component is ready to use. Let's add a route for this component in App.jsx and let's also create a link to it in our navbar:

    1//  previous code if any
    2import MyPreferences from "components/MyPreferences";
    4const App = () => {
    5  //  previous code if any
    6  return (
    7    <Router>
    8      <ToastContainer />
    9      <Switch>
    10        {/* previous code if any */}
    11        <Route exact path="/login" component={Login} />
    12        <Route exact path="/my/preferences" component={MyPreferences} />
    13        {/* previous code if any */}
    14      </Switch>
    15    </Router>
    16  );
    19export default App;

    Now open the Navbar component from components/NavBar/index.jsx and add the following line in order to create a Preferences item in navbar:

    1// previous code if any
    2const NavBar = () => {
    3  // previous code if any
    4  return (
    5    <nav className="bg-white shadow">
    6      <div className="px-2 mx-auto max-w-7xl sm:px-4 lg:px-8">
    7        <div className="flex justify-between h-16">
    8          <div className="flex px-2 lg:px-0">
    9            <div className="hidden lg:flex">
    10              <NavItem name="Todos" path="/dashboard" />
    11              <NavItem
    12                name="Add"
    13                iconClass="ri-add-fill"
    14                path="/tasks/create"
    15              />
    16            </div>
    17          </div>
    18          <div className="flex items-center justify-end gap-x-4">
    19            <span
    20              className="inline-flex items-center px-2 pt-1 text-sm
    21              font-regular leading-5 text-bb-gray-600 text-opacity-50
    22              transition duration-150 ease-in-out border-b-2
    23              border-transparent focus:outline-none
    24              focus:text-bb-gray-700"
    25            >
    26              <Link to="/my/preferences">Preferences </Link>
    27            </span>
    28            <span
    29              className="inline-flex items-center px-2 pt-1 text-sm
    30              font-regular leading-5 text-bb-gray-600 text-opacity-50
    31              transition duration-150 ease-in-out border-b-2
    32              border-transparent focus:outline-none
    33              focus:text-bb-gray-700"
    34            >
    35              {userName}
    36            </span>
    37            <a
    38              onClick={handleLogout}
    39              className="inline-flex items-center px-1 pt-1 text-sm
    40              font-semibold leading-5 text-bb-gray-600 text-opacity-50
    41              transition duration-150 ease-in-out border-b-2
    42              border-transparent hover:text-bb-gray-600 focus:outline-none
    43              focus:text-bb-gray-700 cursor-pointer"
    44            >
    45              LogOut
    46            </a>
    47          </div>
    48        </div>
    49      </div>
    50    </nav>
    51  );

    Implementing task mailer

    Let's create the task mailer by running the following Rails command:

    1bundle exec rails g mailer TaskMailer

    Open the application_mailer.rb from app/mailers/application_mailer.rb and, provide the default email from which the mail will be send:

    1class ApplicationMailer < ActionMailer::Base
    2  default from: ''
    3  layout 'mailer'

    Mailers have methods called "actions" and they use views to structure their content. A controller generates content like say HTML, which will be sent back to the client, whereas a Mailer creates a message to be delivered via email.

    Now, let's edit the task_mailer.rb which we have created.

    Paste the following lines in task_mailer.rb:

    1class TaskMailer < ApplicationMailer
    2  after_action :create_user_notification, if: -> { @receiver }
    4  def pending_tasks_email(receiver_id)
    5    @receiver = User.find_by(id: receiver_id)
    6    return unless @receiver
    8    @tasks = @receiver.assigned_tasks.pending
    9    mail(to:, subject: "Pending Tasks")
    10  end
    12  private
    14    def create_user_notification
    15      @receiver.user_notifications.create(last_notification_sent_date:
    16    end

    As you can see in the above code we have used after_action which is an Action Mailer Callback. Using an after_action callback enables you to perform operations after successfully sending the mail.

    Once we ensure that mail is successfully sent, we will create an entry in user_notifications with today's date, for the current user.

    Our TaskMailer is ready. Let's create a mailer view, which will contain the content and styling of the email.

    Open app/views and you should be able to see the task_mailer directory. Let's create a pending_tasks_email.html.erb file into it:

    1touch app/views/task_mailer/pending_tasks_email.html.erb

    Paste the following lines into it:

    2  <head>
    3    <meta content="text/html; charset=UTF-8" http-equiv="Content-Type" />
    4  </head>
    5  <body>
    6    <h1>Hi! <%= %></h1>
    7    <p>Here is a list of your pending tasks for today.<br /></p>
    8    <ul>
    9      <% @tasks.each do |task| %>
    10      <li><%= task.title %></li>
    11      <% end %>
    12    </ul>
    13  </body>

    Use case of secrets.yml file

    Rails supports the secrets.yml file to store your application's secrets. It is used to store secrets such as access keys for external APIs or our Redis URL etc.

    This will provide reusability and also will help provide us easier access.

    Let's create this file:

    1touch config/secrets.yml

    Open the file and paste the following lines:

    1default: &default
    2  redis_url: <%= ENV['REDISTOGO_URL'] || ENV['REDIS_URL'] || 'redis://localhost:6379/1' %>

    Creating schedule.yml

    We will create a scheduler file containing the cron syntax for different environments and workers.

    Cron is a standard Unix utility that is used to schedule commands for automatic execution at specific intervals.

    Here we are using cron to send an email notification to the users every day at the default time, i.e., at 10 AM or preferred time, if set by the user.

    Let's create the scheduler file:

    1touch config/schedule.yml

    Open the file and paste the following lines into it:

    1default: &default
    2  todo_notifications_worker:
    3    cron: "0 * * * *"
    4    class: "TodoNotificationsWorker"
    5    queue: "default"
    8  todo_notifications_worker:
    9    cron: "* * * * *"
    10    class: "TodoNotificationsWorker"
    11    queue: "default"
    14  <<: *default
    17  <<: *default
    20  <<: *default

    The default and development are two important keys in the YAML file. Each of those keys invoke a worker.

    In the above code, you can see that the cron value for default and development are different. The reason is that in development we don't need to wait. We need the cron to get triggered as soon as possible.

    The cron value for development executes a cron job every minute, whereas for default the same is executed every hour.

    Here in our YAML file, the default key is the main key because its value will be used in other environments like test, staging, and production.

    We create an alias for the default key using an anchor marked with the ampersand & and then reference that alias using the asterisk *. This is purely a "YAML" way of inheriting existing keys.

    Adding logic to schedule cron jobs

    We have already written down when and what needs to be enqueued, in our schedule.yml file.

    Thus in the Sidekiq initializer, we have to add logic that will read the content from our scheduler file and will enqueue based on the cron defined for the specific environment.

    Also, we have to update the REDIS_URL with the URL we have specified in our secrets file.

    Open sidekiq.rb from config/initializers.

    Add the following lines to this file:

    1# frozen_string_literal: true
    3if Rails.env.test? || Rails.env.heroku?
    4  require 'sidekiq/testing'
    5  Sidekiq::Testing.inline!
    10Sidekiq.configure_server do |config|
    11  config.redis = { url:  Rails.application.secrets.redis_url, size: 9 }
    12  unless Rails.env.test? || Rails.env.production?
    13    schedule_file = "config/schedule.yml"
    15    if File.exists?(schedule_file)
    16      Sidekiq::Cron::Job.load_from_hash! YAML.load_file(schedule_file)[Rails.env]
    17    end
    18  end
    21Sidekiq.configure_client do |config|
    22  config.redis = { url: Rails.application.secrets.redis_url, size: 1 }

    We have currently not enabled mail sending in production environment.

    If possible, we will be handling this case in the upcoming chapters.

    Adding TodoNotifications functionality

    For Sidekiq workers, there already exists a generator for generating a worker file.

    Let's create the TodoNotifications worker:

    1rails generate sidekiq:worker TodoNotificationsWorker

    The sole purpose of the TodoNotificationsWorker is process a TodoNotificationsService, which will take care of other core functionalities.

    Open app/workers/todo_notifications_worker.rb and paste the following:

    1# frozen_string_literal: true
    3class TodoNotificationsWorker
    4  include Sidekiq::Worker
    6  def perform
    7    todo_notification_service =
    8    todo_notification_service.process
    9  end

    Let's create our TodoNotificationService to handle the heavy work.

    Rails do not provide any inbuilt generators to create services.

    Currently, we can only manually create the service as well as its test file.

    This service will take care of finding users with pending tasks, whose delivery time falls with the current execution time range, and invoke workers which will send the mail to each user.

    Let's create the service:

    1mkdir -p app/services
    2touch app/services/todo_notification_service.rb

    Open the todo_notification_service.rb file and paste the following:

    1# frozen_string_literal: true
    3class TodoNotificationService
    4  attr_reader :users_to_notify
    6  def initialize
    7    @users_to_notify = get_users_to_notify
    8  end
    10  def process
    11    notify_users
    12  end
    14  private
    16    def get_users_to_notify
    17      users = get_users_with_pending_tasks
    18      users_with_pending_notifications(users)
    19    end
    21    def notify_users
    22      users_to_notify.each do |user|
    23        UserNotificationsWorker.perform_async(
    24      end
    25    end
    27    def get_users_with_pending_tasks
    28      User.includes(:assigned_tasks, :preference).where(
    29        tasks: { progress: 'pending' },
    30        preferences: {
    31          receive_email: true,
    32          notification_delivery_hour: hours_till_now
    33        }
    34      )
    35    end
    37    def users_with_pending_notifications(users)
    38      no_mail_sent = "user_notifications.last_notification_sent_date <>"
    39      first_time_mailing = " is NULL"
    40      delivery_condition = "#{no_mail_sent} ? OR #{first_time_mailing}"
    42      users.left_outer_joins(:user_notifications)
    43        .where(delivery_condition,
    44    end
    46    def hours_till_now
    47      current_hour =
    48      (0..current_hour)
    49    end

    Let's also create a separate worker, named UserNotificationsWorker.

    As per our technical design, we need to ensure that mail delivery for one single user, doesn't break the whole delivery system.

    The first line of defense against this case, is to invoke a separate worker, which is UserNotificationsWorker, for each of the user.

    This way failure of one worker won't affect the other workers, given that Sidekiq runs them in separate threads.

    Let's create UserNotificationsWorker:

    1rails generate sidekiq:worker UserNotificationsWorker

    Open app/workers/user_notifications_worker.rb and paste the following:

    1class UserNotificationsWorker
    2  include Sidekiq::Worker
    4  def perform(user_id)
    5    TaskMailer.delay.pending_tasks_email(user_id)
    6  end

    Phew! That was a lot of content with loads of good stuff.

    That completes our feature and mail delivery should be working in a development environment.

    We will be writing tests for the same, in the next chapter.

    Moving messages to i18n en.locales

    Recall that we already have a Task not found. translation definition inside en.yml file and it is very similar to Preference not found. which is one of the error translation definitions we have to add.

    Rather than having two similar translations for messages which look like record not found, we can use the variable interpolation feature provided by i18n to pass the name of entity which is not found. This will prevent code duplication.

    Fully replace en.yml with the following lines of code:

    2  authorization:
    3    denied: "Access denied. You are not authorized to perform this action."
    4  session:
    5    could_not_auth: "Could not authenticate with the provided credentials."
    6    incorrect_credentials: "Incorrect credentials, try again."
    7  successfully_created: "%{entity} was successfully created!"
    8  successfully_updated: "%{entity} was successfully updated!"
    9  not_found: "%{entity} not found."
    10  task:
    11    slug:
    12      immutable: "is immutable!"
    13  preference:
    14    notification_delivery_hour:
    15      range: "value should be between 0 and 23"
    16    mail:
    17      notification_status: "Email notifications %{status}."
    18  date:
    19    invalid: "must be a valid date"
    20    cant_be_in_past: "can't be in the past"

    Now, the not_found translation in en.yml, will expect a value to be passed for the entity variable. So let's go through our JSON responses and pass that variable with appropriate entity name, wherever we are using the not_found translation key.

    Update the following highlighted line in the load_preference method in PreferencesController:

    1def load_preference
    2  @preference = Preference.find_by(id: params[:id])
    3  unless @preference
    4    render status: :not_found, json: { error: t("not_found", entity:"Preference") }
    5  end

    Also, update the following highlighted line in the load_task method part of the TasksController:

    1def load_task
    2  @task = Task.find_by(slug: params[:slug])
    3  unless @task
    4    render status: :not_found, json: { error: t("not_found", entity: "Task") }
    5  end

    We have also used a Task not found translation inside test_not_found_error_rendered_for_invalid_task_slug in TasksControllerTest where we are testing if the correct error is being rendered for an invalid task slug. Let's update the test case to use the updated i18n translation string.

    1def test_not_found_error_rendered_for_invalid_task_slug
    2  invalid_slug = "invalid-slug"
    4  get task_path(invalid_slug), headers: @creator_headers
    5  assert_response :not_found
    6  assert_equal response.parsed_body["error"], t("not_found", entity: "Task")

    Let's also add the error translations in UserNotification and Preference models.

    Update UserNotification model with the following lines of code:

    1class UserNotification < ApplicationRecord
    2  belongs_to :user
    4  validates :last_notification_sent_date, presence: true
    5  validate :last_notification_sent_date_is_valid_date
    6  validate :last_notification_sent_date_cannot_be_in_the_past
    8  private
    10    def last_notification_sent_date_is_valid_date
    11      if last_notification_sent_date.present?
    12        Date.parse(last_notification_sent_date.to_s)
    13      end
    14    rescue ArgumentError
    15      errors.add(:last_notification_sent_date, t("date.invalid"))
    16    end
    18    def last_notification_sent_date_cannot_be_in_the_past
    19      if last_notification_sent_date.present? && last_notification_sent_date <
    20        errors.add(:last_notification_sent_date, t("date.cant_be_in_past"))
    21      end
    22    end

    Update the following highlighted line in the Preference model:

    1class Preference < ApplicationRecord
    2  belongs_to :user
    4  validates :notification_delivery_hour,
    5    presence: true,
    6    numericality: { only_integer: true },
    7    inclusion: {
    8      in: 0..23,
    9      message: I18n.t("preference.notification_delivery_hour.range")
    10    }

    Testing PreferencesController

    To test the preferences controller, we do not need to create a preference factory since a preference association is created automatically for each user when that new user is created.

    Now, create a new file test/controllers/preferences_controller_test.rb and add the following content:

    1require "test_helper"
    3class PreferencesControllerTest < ActionDispatch::IntegrationTest
    4  def setup
    5    @user = create(:user)
    6    @preference = @user.preference
    7    @headers = headers(@user)
    8  end

    Now, let us test if the correct preferences are rendered for a valid user. Append the following test case to the PreferencesControllerTest:

    1def test_show_preference_for_a_valid_user
    2  get preference_path, headers: @headers
    3  assert_response :ok
    4  assert_equal response.parsed_body["preference"]["id"],

    Let us test if the update action is working as intended. Add the following test cases to the PreferencesControllerTest:

    1def test_update_success
    2  preference_params = { preference: { receive_email: false } }
    4  put preference_path, params: preference_params, headers: @headers
    5  @preference.reload
    6  assert_response :ok
    7  refute @preference.receive_email
    10def test_update_failure_for_invalid_notification_delivery_hour
    11  preference_params = { preference: { notification_delivery_hour: 24 } }
    13  put preference_path, params: preference_params, headers: @headers
    14  assert_response :unprocessable_entity
    15  assert_equal response.parsed_body["error"], "Notification delivery hour #{t("preference.notification_delivery_hour.range")}"

    Let us test the mail action. Add the following test case to the PreferencesControllerTest:

    1def test_update_success_mail
    2    preference_params = { preference: { receive_email: false } }
    4    patch mail_preference_path, params: preference_params, headers: @headers
    5    @preference.reload
    6    assert_response :ok
    7    refute @preference.receive_email
    8  end

    Now you can try running all the preference test cases and they should be passing:

    1bundle exec rails test -v test/controllers/preferences_controller_test.rb

    Now let's commit these changes:

    1git add -A
    2git commit -m "Added ability to send email to each user with pending tasks"