That's where Crunchy Data comes in. They providemanaged Postgres service. Checkout thistweet from DHH about CrunchyData.

In our internal discussion, one of the BigBinary engineers brought up the issueof "latency". Since the PostgreSQL server will not be in the same data center,what would be the latency? How much impact will it have on performance?

We didn't know the answer so we thought we would do some benchmarking.

Benchmarking

To do the comparison we needed another hosting provider where we can runPostgreSQL on the same data center. We chose to work with Digital Ocean.

To compare the two services, we wrote a benchmark script in Ruby similar to theone Ben Johnson wrote inGofor his GopherCon talkin 2021.

In this benchmark, we're using Ruby's Benchmark module to measure theperformance of a series of database operations. Here's what the code does:

1. It establishes a connection to the database only once, at the beginning ofthe script. This is done outside the benchmarked operations becauseestablishing connections can be a slow operation because of TLS negotiation,and we don't want to account for that time in our measurements.

2. It then performs the following operations 10,000 times, measuring each one:

   • Drops a table named 't' if it exists.
   • Creates a new table 't' with two columns: 'id' (an auto-incrementingprimary key) and 'name' (a text field).
   • Inserts a single row into the table with the name 'jane'.
   • Selects the 'name' from the table where the 'id' is 1 (which should be'jane').

3. After all 10,000 iterations, it calculates and prints the average time foreach operation in microseconds.

require "pg"require "benchmark"class PostgresBenchmark  def initialize(connection_string)    @conn = PG.connect(connection_string)  end  def run(iterations = 10_000)    total_times = Hash.new { |h, k| h[k] = 0 }    iterations.times do |i|      puts "Running iteration #{i + 1}" if (i + 1) % 1000 == 0      times = benchmark_operations      times.each { |key, time| total_times[key] += time }    end    average_times = total_times.transform_values { |time| time / iterations }    print_results(average_times, iterations)  ensure    @conn.close if @conn  end  private  def benchmark_operations    times = {}    times[:drop] = Benchmark.measure { @conn.exec("DROP TABLE IF EXISTS t") }.real    times[:create] = Benchmark.measure { @conn.exec("CREATE TABLE t (id SERIAL PRIMARY KEY, name TEXT)") }.real    times[:insert] = Benchmark.measure { @conn.exec("INSERT INTO t (name) VALUES ('jane')") }.real    times[:select] = Benchmark.measure do      result = @conn.exec("SELECT name FROM t WHERE id = 1")      raise "Unexpected result" unless result[0]["name"] == "jane"    end.real    times  end  def print_results(times, iterations)    total_time = times.values.sum    puts "\nAVERAGE ELAPSED TIME (over #{iterations} iterations)"    puts "drop    #{(times[:drop] * 1_000_000).round(2)} microseconds"    puts "create  #{(times[:create] * 1_000_000).round(2)} microseconds"    puts "insert  #{(times[:insert] * 1_000_000).round(2)} microseconds"    puts "select  #{(times[:select] * 1_000_000).round(2)} microseconds"    puts "TOTAL   #{(total_time * 1_000_000).round(2)} microseconds"  endendif __FILE__ == $0  connection_string = "<DB_CONNECTION_STRING>"  benchmark = PostgresBenchmark.new(connection_string)  benchmark.run(10_000)end

Database Specifications and Setup

Digital Ocean

• Region: NYC3 data center
• Specs: 2 vCPU, 4GB Memory
• Price: $60 per month

Crunchy Data

• Provider: AWS
• Region: us-east-1
• Specs: 2 vCPU, 4GB Memory (hobby-4 plan)
• Price: $70 per month

We provisioned a Digital Ocean droplet in the NYC3 data center and invoked thebenchmark script from the machine. The Digital Ocean database was also in thesame NYC3 data center. For Crunchy Data, the availability zone selected wasus-east-1 as it was the closest to NYC3.

Benchmarking results

Digital Ocean

AVERAGE ELAPSED TIME (over 10000 iterations)drop    3448.07 microsecondscreate  5048.39 microsecondsinsert  891.81 microsecondsselect  584.17 microsecondsTOTAL   9972.44 microseconds

Crunchy Data

AVERAGE ELAPSED TIME (over 10000 iterations)drop    10097.89 microsecondscreate  16818.63 microsecondsinsert  8416.35 microsecondsselect  7211.42 microsecondsTOTAL   42544.29 microseconds

Benchmarking analysis

The results of this benchmark do not come as a surprise. Digital Ocean isperforming significantly better than Crunchy Data. This performance differencecan be primarily attributed to network latency.

Network latency refers to the round-trip time (RTT) it takes for the data totravel from its source to its destination and back again across a network. Inthe context of database operations, it's the time taken for a query to be sentfrom the client to the database server and for the response to return to theclient.

In our benchmarking, the Digital Ocean database and the client machine invokingthe script were both located in the same data center (NYC3), resulting inminimal network latency. On the other hand, the Crunchy Data database was hostedin AWS us-east-1, and it had to communicate across a greater physicaldistance, adding to latency.

To get a more accurate value for the network latency, we can compare the averagetime taken to run the SELECT operation. The SELECT operation in the scriptis a point query. A point query refers to type of query that retrieves one orseveral rows based on a unique key.

In our script, it retrieves a single name value from the table t where theid is 1(which is the primary key), and is very fast to execute. Thus, thetime taken to execute the SELECT operation can give us an approximate valuefor the network latency.

db_time = network_latency + query_execution_time

For point queries, the query_execution_time is almost zero so all the timetaken is pretty much "network latency".

db_time  network_latency

If we look at the benchmarking result, then we can see that for "select"operation time taken by Digital Ocean is "584 microseconds" and for Crunchy Datait is "7211 microseconds".

The difference in network latency is 6627 microseconds. That is 6.6milliseconds.

This value over multiple queries can add up and can have a significant impact onthe overall response time of your application. To put this into perspective, fora web application that makes 10 sequential database queries to render a page,this could add up to about 66 milliseconds to the page load time. Now, thiscould be an acceptable limit if your page loads in 3/4 seconds. However,if you are trying to load your page in 200 millisecond,s then you need to watchout.

Ensuring that the database is always up and it's properly backed up is a non-trivialproblem. Latency notwithstanding, Crunchy Data takes care of running the database.This gives us peace of mind and allows us to exit the cloud one step at a time.

The Problem

Initially, when we shared a NeetoRecord video on Twitter, this is how the tweetwould appear.

As we can see, the links are just plain text, lacking visual appeal and context.Users had to click away from Twitter and open a new tab to view the recording.This extra step not only disrupted the user's Twitter experience but also likelyreduced the number of people who actually watched our videos.

The Solution: Twitter Cards

To address this issue, we turned to Twitter Cards. Similar to Facebook's OpenGraph protocol, Twitter Cards allows us to showcase interactive media directlywithin tweets. While both protocols serve similar purposes, Twitter cards aredesigned to directly work in Twitter, so its crawler will look for these tagsand only look for OG tags as a fallback.

What are Twitter Cards?

Twitter Cards are a set of meta tags that enable us to embed interactive contenton a tweet. In our case, we want to provide users an embedded video player whichthey can use to view the NeetoRecord recording without leaving Twitter. It's notjust about aesthetics. According to Twitter's own data, "tweets with cards have43% higher engagement rates than regular tweets with links." This statisticunderscores the impact of providing a seamless, visually appealing experience.

Implementing Twitter Cards for NeetoRecord

There are multiple types of cards that we can leverage to enhance our contentsuch as summary cards, player cards & app cards. However, in this post, we aregoing to be discussing about Player cards, which is implemented in NeetoRecord.This card type allows us to embed an interactive video player directly in thetweet. Here's how it looks:

Clicking on this link preview will open our embedded content, in this case, aplayer to view the recording directly within Twitter.

As we can see, users can now watch our NeetoRecord videos without leavingTwitter. This seamless experience has several benefits:

• Higher Engagement: With the video right there in the tweet, more users arelikely to watch it.
• Reduced Friction: No more clicking away or opening new tabs, keeping usersengaged with our content and the Twitter conversation.
• Better Branding: The Player Card includes our logo and video title,reinforcing our brand with every view.

Technical Implementation

Implementing Twitter Cards is straightforward. We added meta tags to the<head> section of our web page. For our Player Card, we use tags as shownbelow.

<meta property="twitter:card" content="player" /><meta  property="twitter:url"  content="https://oli.neetorecord.com/watch/864de7fb-2efb-4f2f-a60b-08dca64e4c3"/><meta  property="twitter:title"  content="Introducing the new video player for NeetoRecord"/><meta property="twitter:site" content="@NeetoRecord" /><meta property="twitter:image" content="https://cdn.neeto.com/hycoe7" /><meta  property="twitter:player"  content="https://oli.neetorecord.com/embeds/864de7fb-2efb-4f2f-a60b-08dca64e4c3"/><meta property="twitter:player:width" content="1280" /><meta property="twitter:player:height" content="720" />

These tags tell Twitter's crawler what to display in the card.

Let's break down the purpose and importance of each meta tag:

<meta property="twitter:card" content="player" />

This tag specifies the card type. Setting it to "player" tells Twitter that thisis a Player Card, which is designed for video or audio content.

<meta  property="twitter:url"  content="https://oli.neetorecord.com/watch/864de7fb-2efb-4f2f-a60b-08dca64e4c3"/>

This tag provides the URL of the web page that the card is describing. It shouldbe the page where users can view the full content.

<meta  property="twitter:title"  content="Introducing the new video player for NeetoRecord"/>

This tag sets the title of the card, which appears as the main headline.

<meta property="twitter:site" content="@NeetoRecord" />

This tag specifies the Twitter @username the card should be attributed to.

<meta property="twitter:image" content="https://cdn.neeto.com/hycoe7" />

This tag provides the image to be displayed in place of the player on platformsthat dont support iFrames or inline players.

<meta  property="twitter:player"  content="https://oli.neetorecord.com/embeds/864de7fb-2efb-4f2f-a60b-08dca64e4c3"/>

This crucial tag specifies the URL of the video player. This should be an HTTPSURL to an iframe player that can play the content.

<meta property="twitter:player:width" content="1280" /><meta property="twitter:player:height" content="720" />

These two tags define the width and height of the video player iframe, inpixels. They help ensure the video displays correctly in the Twitter feed.

Conclusion

Twitter Cards have improved how NeetoRecord videos are shared on Twitter. Theembedded media experience reduces friction and increases engagement, whileenhancing our brand visibility.

For our apps deployed on Heroku, we used the following command to get the latestcommit hash as an environment variable in the Heroku server on each build:

heroku labs:enable runtime-dyno-metadata -a your-app-name

With NeetoDeploy, we didn't need this stepsince git is available in the console and we used git commands to set the latestcommit hash as the Honeybadger revision. Honeybadger uses therevisionto associate maps with the bundle when the bundle name doesn't change.

Steps to integrate Honeybadger:

1. Create a new project in the Honeybadger dashboard. We've made avideo on setting it up following theReact integration guide.

2. Set the environment variable HONEYBADGER_JS_API_KEY with the Honeybadgerproject's API key. The API key can be copied from Settings => API Keys.

In depth guides for Honeybadger integration & sourcemap upload:

• What is honeybadger
• Honeybadger integration basics
• Honeybadger integration in the Neeto ecosystem
• Sourcemaps explained
• Uploading sourcemap to honeybadger
• Sourcemaps in action within Honeybadger errors
• Handling CDN based application bundles

We've created a YouTubeplaylistof the above videos for easy access.

While writing the program, we realized that we'd need a switch-case that checkedwhat data type each key was. We had to search for the get and set methods foreach data type manually while writing the program. Late,r we used the redis-rbgem and was able to get the dump with OpenSSL::SSL::VERIFY_NONE, but we hadstill faced the issue of having to look up what the command was for differentdata types like zset and hash.

I thought this could be a good opportunity to write a blog post summarizing allthe data types and the commands associated with them. The Redis documentationhas a page about the different data types, but all the commands are notavailable at a single place.

Redis key-value database

When we say that Redis is a key-value database, there is more to it than whatmeets the eye. Redis keys can store values of several different data types.Redis has a set of commands for each of these different data types to dooperations with them. In this post, well go over the different data types, whatthey are and how we can work with them.

What are the different data types?

Redis has more than a couple of data types, which can be used to store differentdata based on your needs. These include the following:

• String - The basic data type we are all familiar with.
• List - An array of strings.
• Hash - A collection of key-value pairs, similar to a Ruby Hash.
• Set - A collection of unique strings.
• Sorted Set - A collection of unique strings maintaining by each stringsscore.
• HyperLogLog - Probabilistic estimates of cardinality of large sets.
• Stream - An append only log.
• Geospatial Index - Data structure for storing geographic coordinates.

Checking data type of your keys

You can use the TYPE command to check what data type your key is. Once youknow what type your key is, you can use the commands associated with your key'sdata type to interact with it.

127.0.0.1:6379> TYPE schedulezset

Cheatsheet for Redis commands based on data type

Each Redis data type has its own set of commands for doing operations with thekey and its value. Here's a quick overview of the basic data types you wouldencounter and some of the basic commands to set, retrieve and delete data fromthem.

Read more about the different data types in Redis and all the different commandsthat are available in theirofficial documentation.

zprof

zprofis a utility that comes packaged with zsh, which you can use to profile your zshscript.

Add the following to the top of your .zshrc file to load zprof.

zmodload zsh/zprof

At the bottom of your .zshrc, add the following.

zprof

This would profile your zsh script and print a summary of all the commands runduring your shell startup and the time it takes to execute them. Run exec zshto apply the changes and restart your shell. Your shell will print somethinglike this:

With this you can see which commands are taking the most time to load. Enablingprofiling has helped pinpoint the issue and now you can look into fixing it. Inthe above example, you can see that nvm is taking up a considerable amount oftime.

When it comes to actually using these products, you will be redirected tohttps://subdomain.neetocal.com. Here, "subdomain" would be the subdomainallocated to you when you signed up for Neeto.

We planned to add "Google sign in" feature to make it easier for folks toboth signup and to log in. During testing, it all worked fine. However, when weasked "Google" to approve the app "NeetoCal" for "Google sign in" Googledemanded that our users should be able to see the "Privacy Policy" and "Terms ofconditions" on the website. In order to make Google happy, we added a redirectionfrom "neetocal.com" to "neeto.com/neetocal".

However, Google was not happy with it. The users are logging intohttps://subdomain.neetocal.com so the "privacy policy" and "terms of service"should be visible on the domain "neetocal.com" itself.

We are using Cloudflare as our DNS provider. Using the tools provided to us byCloudflare we decided to show the content of "neeto.com/neetocal" on"neetocal.com" without redirecting the user.

Note that in this case, if you type "neetocal.com", then you will see the URLchange to "neetocal.com/neetocal" instantly. That's because the URL of the mainmarketing site is "neeto.com/neetocal".

Cloudflare providesPage ruleswhich we will be using to achieve our goals. Below is a video of how it was done.

<iframewidth="100%"height="315"src="https://www.youtube.com/embed/RodZIBxYBHc"frameborder="0"allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"allowfullscreen

</iframe>

SEO duplicate content issue

The Google search engine doesn't like it when we show exactly the same contenton two different domains. Google thinks that the site is trying to cheat Googleand Google will punish both sites.

We want Google to index our main marketing site https://neeto.com/neetocal andwe want Google to ignore "neetocal.com". One way to tell Google not index thesite is by adding a noindexmeta tag.

<meta name="robots" content="noindex">

In the above example, we are asking all bots not to index the page containingthe above meta tag.

We planned to inject this meta tag when a page is rendered for the URL"neetocal.com" and we will not inject this tag when the page is rendered for the URL"neeto.com".

Upon more research, we found that search engines also look at the responseheaders. Given below is the sequence that search engines follow for indexing theweb pages.

• Crawler gets the raw page source as a response to the HTTP request.
• Crawler checks if x-robots-tag: noindex, nofollow header is present in theresponse.
• Crawler checks the meta tags to determine if the page needs to be indexed ornot.

If a page has x-robot-tag: noidex, nofollow then the crawler will not indexthe page.

Based on this information, we decided to useResponse Header Modification Rulesfeature of Cloudflare. Below is a video of how it was done.

<iframewidth="100%"height="315"src="https://www.youtube.com/embed/0AFeM2yyg_A"frameborder="0"allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"allowfullscreen

</iframe>

<iframewidth="560"height="315"src="https://www.youtube.com/embed/eS4BAhk7DAo"title="Debugging high GitHub action usage"frameborder="0"allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"allowfullscreen

</iframe>

We checked with SSLlabs and it said the same thing: the servers support TLS 1.0and TLS 1.1.

TLS 1.0 was published in 1999, and TLS 1.1 was published in 2006. Microsoft andother companies don't support these two versions of TLS. Even Herokudoesn't supportit.

All our Neeto applications are hosted on Heroku. If Heroku doesn't support TLS1.0 and TLS 1.1, how come the server supports these older versions of TLS?

Solving the TLS issue using Cloudflare

We use Cloudflare as our DNS server for all Neetoproducts. Cloudflare allows us to proxy the request. It means that when the userhits neetocal.com, their request is not going to Heroku. Cloudflare willintercept the request, and then Cloudflare will make a request to the Herokuserver on behalf of the user. When Cloudflare makes this request to Heroku willuse its own SSL certificate.

Cloudflare allows us to have control over the "Minimum TLS version" to support.We configured Cloudflare to not support TLS 1.0 and TLS 1.1.

The following video goes into step-by-step detail on how we configured this inCloudflare.

<iframewidth="560"height="315"src="https://www.youtube.com/embed/sED8_Qwmi2w"title="Upgrading to TLS 1.2 using Cloudflare"frameborder="0"allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"allowfullscreen

</iframe>

CDN77 is the service we used in the video tocheck the TLS version.

We switched our pull-requestreview apps from Herokuto NeetoDeploy a couple of months ago, and it has been doing well. As the nextstep of the process, we are building features to target staging and production.We had an essential staging set up in March, and by the end of the month, we hadmigrated staging deployments of all Neeto from Heroku to NeetoDeploy. Everythingwas working fine for a week until I made a grave mistake.

What happened

Whenever a PR is opened or a new commit is pushed, NeetoDeploy receives awebhook call from GitHub. Review/staging apps are created/updated on NeetoDeployin response to these webhook calls.

On 2023 April 5, a misconfiguration caused NeetoDeploy's webhook handler tomalfunction for an hour. As a result, some review apps were not deleted evenafter the corresponding PRs were closed or merged. The solution wascross-checking review apps with the open PRs and deleting the unwanted apps.Using rails console, this could be done live on the server.

Here is what that solution looked like:

GithubRepository.find_each do |github_repository|  access_token = github_repository.github_integration.access_token  github_client = Octokit::Client.new(access_token:)  open_pr_numbers = github_client    .pull_requests(github_repository.name, state: :open)    .pluck(:number)  github_repository.project.apps.find_each do |app|    next if open_pr_numbers.include?(app.pr_number)    Apps::DestroyService.new(app).process!  endend

But there is a terrible mistake in the above code. See if you can spot that.

I'm going to wait...

A bit more waiting... Enough waiting; here's the mistake:

There is no filter in the apps that were picked to be destroyed. This snippetwas written at a time when we only had review apps. Sogithub_repository.project.apps was expected to return review apps. But we nowalso had staging apps in the database. And those staging apps weren't filteredout here. After running the snippet and noticing it took longer than expected, Irealized the mistake and instantly pressedCTRL + C. Of course, it was takingtime since it was deleting all the staging app databases and dynos .

In the end, out of 33 staging apps, only five remained. And thus started, theprocedure to restore all of them.

The recovery

NeetoDeploy already had the feature to do manualDB exports but thiswasn't being done routinely. We were only hosting review apps (whose data neednot be persisted reliably), and staging had only started just a week before.

We had database backups from a week before (when we ultimately migrated stagingapps off Heroku), and one by one, our small team of 4 brought back all the appsin 2 days. The next step is to try not to let this happen again; if it were tohappen, we have a contingency plan. We thought of two types of contingencyplans:

• Automatic scheduled backups
• Disk snapshots of the DB

Automatic scheduled backups

The idea is that the database would be exported at a particular time every day.Backups older than a month would be deleted automatically to save space.

We implemented this in a week. Every day at 12 AM UTC, all staging+productiondatabases would be exported and uploaded to an S3 bucket.

While this feature was being implemented, I used the Rails console to manuallyexport all the apps. The exported file URLs of each DB were manuallycopied to a text file. aria2c was then used todownload them in parallel to a local folder:

aria2c -c --input-file export_urls.txt

aria2c is a smart downloader. It will resume interrupted downloads, wouldntduplicate downloads, and do everything in parallel.

Disk snapshots of the DB

The other contingency method is to do periodic snapshots of the volume holdingthe DB. We are working on this.

You can refer to thisblog post of GitLabto know their recovery procedures when they faced a significant data lossin 2017.

Lessons

The core lesson here is to call destructive methods very carefully. Instead ofcalling the DestroyService instantly, there could have been an intermediatehuman check:

apps = []GithubRepository.find_each do |github_repository|  access_token = github_repository.github_integration.access_token  github_client = Octokit::Client.new(access_token:)  open_pr_numbers = github_client    .pull_requests(github_repository.name, state: :open)    .pluck(:number)  github_repository.project.apps.review.find_each do |app|    next if open_pr_numbers.include?(app.pr_number)    apps.append(app)  endend

This would populate the list of apps to delete in apps variable, it can bedisplayed, verified and then we can destroy them individually:

apps.map do |app|  Apps::DestroyService.new(app).process!end

The other takeaway here is to have proper recovery mechanisms in place.Human/system errors are possible; we should be prepared when it happens.

NeetoDeploy is still not production-ready.However, if you want to give NeetoDeploy a try, then tweet to us at@neetoDeploy or send us an email atinvite@neeto.com.

Not only can anyone send an email to anyone, but anyone can pretend to be anyone. Ican send an email to you and the email will come to you as if it were sent by"Elon Musk" and the from address could be "elon@musk.com".

I can send an email to you pretending that the email is from your bank and youneed to change your password. In one case, a hacker sent an email to a company'sfinance team telling them that the company's bank account number has changed.The hacker walked away with millions of dollars.

Hackers can also do damage to one's domain reputation.

Let's assume that there is a business called "PrixPapers" and they havethousands of customers all over the country. Malicious folks can start sendingemails to people pretending to be "PrixPapers". When people receive an emaillike this they might buy the item. The item could be "fake goods". Ideally, noone should be able to send any email pretending to be someone else.

Since these malicious folks don't have the actual list of the customers ofPrixPapers will in general spam people. Some people receiving this email willmark these emails as "spam". As more and more people mark these emails as "spam"the domain reputation of "prixpapers.com" will go down. It means that when"prixpapers.com" will send out a legitimate newsletter to its customers some ofthese emails will be marked as "spam" because of the nefarious work done by theprevious scammer.

The challenge presented to the Internet authorities is that they want to keepemail simple but safe at the same time. They came up with certain policies tomake emails safe and secure. In this blog, we will see how SPF helps in keepingemail secure. However, first, we need to know what is "Return-path". We'll latersee how it is used in blocking fake emails.

Return-path

Let's say that we are dealing with a company calledNeetoCal. They have the domain neetocal.com.

Let's say that brian@neetocal.com sent an email to peter@gmail.com. Let'spretend that for some reason Peter's mailbox is full and the email sent by Brianbounces. In that case, Brain will get an email that the message bounced. We allhave seen these kinds of messages.

Now let's imagine that it's end of the year sale time and NeetoCal decided tosend an email campaign to all its customers giving them a 10% discount. Thisemail will go out to their 5000 customers. Some of these emails are bound tobounce. If all the bounced emails come to Brain then Brain's inbox would befilled with such bounced emails.

To solve this problem email protocol allows us to set a hidden field calledReturn-path. This value can be set in the email header. Return-path set inthe email header indicates how to process bounced emails. Anytime an emailserver detects that an email can't be delivered for reasons like "mailbox isfull" or "the email doesn't exist" then that email server can send an email tothe email address mentioned in Return-path. The business owner can look atReturn-path emails to analyze how many emails are bouncing and why.

If you are sending an email using "Gmail", "outlook", "yahoo" etc then theReturn-path is set as your email so that you get to know if there is a bouncedemail. Email service providers allow us to customize Return-path in case oneis running a big marketing campaign. For example here is adocument from SendGriddescribing how to set Return-path.

Why are we talking about how bounced emails are processed when we are dealingwith the subject of "email deliverability". That's because Return-path plays adual role as we will see next.

What is SPF record

Before we get into the SPF record, let's take a simple real-world example of how itworks.

Let's say there's a gatekeeper at the NeetoCal office. This gatekeeperwill only allow the people who work there. It means the gatekeeper has a list ofapproved people and for each person getting through the gate, the gatekeeperchecks if the person is in the approved list or not.

SPF works similarly. SPF policy is a mechanism to tell the email server if theemail is coming from a trusted source, then accept the email or reject the email.A sender policy framework (SPF) record is a type of DNS TXT record that listsall the servers authorized to send emails from a particular domain.

Let's take a real-world example. The SPF record of neetocal.com looks likev=spf1 include:spf.messagingengine.com -all. We can see this datausing mxtoolbox.

How the SPF record is used

Let's look at the following case.

Step 1. notifications@neetocal.com sends an email to elon@gmail.com.

Step 2. This email is sent to Fastmail since NeetoCal isusing the Fastmail services.

Step 3. The Fastmail email server receives this email as an outgoing email.

Step 4. Fastmail email server adds email header Return-path value to"notifiations@neetocal.com".

Step 5. The Fastmail email server sends this email to the gmail server.

Step 6. The Gmail server gets this email.

Step 7. The Gmail server extracts the Return-path key and finds that thedomain is "neetocal.com".

Step 8. The Gmail server finds the TXT DNS records of "neetocal.com". This listsall the approved IP addresses.

Step 9. The Gmail server will check if the email server which sent the email isin the approved IP addresses or not.

Step 10. If the IP address is in the approved IP addresses list then the emailis approved for further processing. Otherwise, the email is rejected.

We can go to https://dmarcian.com/spf-survey/ and enter "neetocal.com" here.They get the IP addresses published for "messagingengine.com" and then they showthe list of the approved IPs.

Allowing the third party to send emails on your behalf

Let's say that Brian decides to use Mailerlite tosend marketing emails. Now if Mailerlite sends an email then the receiving emailserver will notice that the IP address of Mailerlite is not in the approved listof ips and the email will be rejected.

A domain is allowed to have only one SPF record. So if NeetoCal wants to useMailerlite for marketing then the spf records need to be updated. If you signupfor a domain in Mailerlite then Mailerlite will check if that domain has anexisting SPF record or not. If there is no SPF record then Mailerlite won't doanything. However, if there is an existing SPF record then Mailerlite willinsist that first you update the SPF record to include Mailerlite so that theemails from Mailerlite are not rejected.

BigBinary uses Mailerlite to email newly publishedblogs to the subscribers. Given below is what the SPF record of BigBinary lookslike. We can also see this resultonline.

v=spf1 include:_spf.mlsend.com include:_spf.google.com -all

BigBinary uses google workspace so the second include is for that reason. Thefirst include is to ensure that Mailerlite is in the allowed ip list.

If we look at the result for "bigbinary.com" by visitinghttps://dmarcian.com/spf-survey/ then we will that all these includes are likeregular programming language "imports". They allow the third party to includeother third parties in the chain. The end result is that we have a finite listof approved IP addresses which can send email.

spf record standard

Let's take a look at NeetoCal's SPF record.

v=spf1 include:spf.messagingengine.com -all

v=spf1 tells the server that this record contains an SPF record. Every SPFrecord must begin with this string.

include:spf.messagingengine.com tells the server what third-partyorganizations are authorized to send emails on behalf of the domain. This tagsignals that the content of the SPF record for the included domain(messagingengine.com in this case) should be checked and the IP addresses itcontains should also be considered authorized. Multiple domains can be includedwithin an SPF record.

-all tells the server that addresses not listed in the SPF record is notauthorized to send emails and should be rejected. Alternative options hereinclude ~all, which states that unlisted emails will be marked as insecure orspam but still accepted. +all signifies that any server can send emails onbehalf of the domain.

Here is another format of SPF record.

v=spf1 ip4=192.0.2.0 ip4=192.0.2.1 include:messagingengine.email -all

In this example, the SPF record is telling the server that ip4=192.0.2.0 andip4=192.0.2.1 are also authorized to send emails on behalf of the domain.

All this time, the AceInvoice website was not redirecting to NeetoInvoice. Todaywe did that using Cloudflare. The URL forwarding orredirecting withpage ruleis a neat feature of Cloudflare. Below are the screenshots of the steps taken toredirect the URLs. More details are covered in the video.

Handling www version

Handling no www version

<iframewidth="560"height="315"src="https://www.youtube.com/embed/R6-qnYN6PUs"title="YouTube video player"frameborder="0"allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"allowfullscreen

</iframe>

We didn't mention it in the video, but it's worth knowing that we need to have aCNAME record for a subdomain that needs forwarding. For examples let's saythat we need to forward all traffic from https://videos.bigbinary.com tohttps://bigbinary.com/video. We can't add a page rule for this directly. Forwe need to add a DNS entry for subdomain videos and this entry must havecloudflare "proxy" checked so that you see Proxied next to it. If you see "DNSonly" then that means Cloudflare will not be able to do any forwarding.

OAuth 2.0

OAuth 2.0 is an authorization framework that enables an application to accessdata from a server on behalf of a user. For example, OAuth enables applicationsto access data from Google, Facebook, GitHub, etc., on behalf of users havingaccounts in that service.

Let's say you have an application that requires access to a user's data onGitHub. The following are the steps involved in authorizing your applicationwith GitHub:

1. The application requests authorization from GitHub. The following are some ofthe parameters this request should contain:

   • client_id - This is an ID used by GitHub for identifying the application.You need to register your application with GitHub to get this ID.
   • redirect_uri - The URI to which GitHub should send back a request oncethe authorization is approved.
   • login - This is a username on GitHub. The application requires access todata on behalf of the user with this username.
   • state - This should ideally be a string of random characters. Store thisstring in memory because it will be used in another step. This parameter isoptional but highly recommended. We'll look into why later.

2. GitHub then asks the user to grant the authorization. The user might beprompted to log in to GitHub if they have not already logged in. Then GitHubdisplays information on the application and lists all the data theapplication wants to access. If the user denies authorization, the processends here. If the user approves, we move on to the next step.

3. GitHub sends a request to the application through the URI passed asredirect_uri in the first step. This request will contain a codeparameter that serves as a temporary authorization code and a stateparameter.

4. The OAuth process must be dropped immediately if the value of the stateparameter from the previous step does not match the random string stored inmemory in step 1. This ensures that the OAuth process was initiated by yourapplication. We'll look more into this later.

5. The application should send another request to GitHub to generate a permanentauthentication token. This request should contain the code sent by GitHubin step 3.

6. GitHub responds with an authentication token that can be used by theapplication to access data on behalf of the user.

The state parameter

The state parameter plays a crucial role in ensuring that the GitHub OAuthprocess was initiated by your application or a trusted source. An unauthorizedthird party could pose as your application by using your client_id. The valueof client_id is not exactly a secret. GitHub treats any OAuth processinitiated using your ID as an OAuth process started by your application.

Let's see how the OAuth process would play out when initiated by an unauthorizedthird party:

1. Third-party requests authorization from GitHub. The third party would passyour ID as client_id. This would lead GitHub to believe the request is fromyour application. The third party may or may not pass a state parameter.

2. GitHub then asks the user to grant the authorization. If the third party usesone of their own accounts, they could grant the authorization. The thirdparty could also convince a user to grant permission using socialengineering. For example, they could perform a phishing attack using an emaildesigned to trick the user into believing that the email was from yourapplication.

3. GitHub sends a request to the application through the URI passed asredirect_uri in the first step. This request would contain the codeparameter, and the state parameter if it was passed in step 1.

4. It would be clear that the process was not initiated by your application ifthe state parameter is missing. Even if there was a state parameter, itwould not be found in memory. This is because the state parameter wasgenerated by a third party and not your application. Only those stateparameters generated by your application will be found in your memory.

JSON Web Tokens

JSON Web Tokens (JWT) is a structured format that contains header, payload, andsignature components. Let's take a look at the significance of these components:

1. Header - The header is a JSON string that typically contains the signingalgorithm used, such as HMAC, SHA256, or RSA.

2. Payload - The payload is a JSON string that contains claims andadditional data. Claims can be used to enforce security constraints andvalidate the authenticity of the token and its contents. For example, aclaims can specify the token expiration time. The expiration time, mostcommonly represented by exp, represents the date and time after which thetoken will no longer be considered valid.

3. Signature - The signature is used to verify that the sender of the JWT iswho it says it is and to ensure that the token has not been tampered withalong the way. Just like the hand signature of a person, it is hard to forgea JWT signature. (Digital signatures are significantly harder to forgecompared to a person's hand signature)

Generating a JWT

The following steps outline the process of generating a JWT:

1. The header and the payload of the JWT are first encoded using Base64encoding. So now we have two strings - the encoded header and the encodedpayload.

2. The encoded header and the encoded payload are concatenated into a singlestring, with dots (.) separating each part. The concatenated string is signedusing the signing algorithm specified in the header. The signing algorithmuses a secret key, that is known only to the issuer (your application), whichensures that only the issuer can generate a valid signature. The result ofthe signing process is a signature, which is also a string.

3. The encoded header, the encoded payload, and the signature are concatenatedinto a single string, with dots (.) separating each part. The resultingstring is the JWT, which can be transferred securely between parties.

For example, let's say we have the following header:

{ "alg": "RS256" }

The value of alg contains the algorithm used for signing the JWT. Also, let'ssay we have the following payload:

{ "username": "sam@example.com", "exp": 1676263763 }

The username is data that needs to be transferred. exp contains theexpiration time claim of the JWT.

When both these components are encoded, we get:

• Header - "eyJhbGciOiJSUzI1NiJ9"
• Payload - "eyJ1c2VyIjoic2FtQGV4YW1wbGUuY29tIiwiZXhwIjoxNjc2MjYzNzYzfQ"

When these are signed using the RS256 algorithm and a secret key, a signature isproduced. The following signature was generated using the RS256 algorithm and asecret key (which will remain secret):

NlAT6awp68dCEcFXbDeeLTzZekqUmB3f6kr3jkGSFmrKa5zvLmFGeraWba_fUuQLVhRtcXUPZbRR1DKnKH0HVf1rRDvOqezwbhe-hR1wlz6vZkHuPjtYSCLx_aybGm7dy2ijfTQwYd14cD9ZiMI5vf6XcDDfE7mkhu0ogCOnqR1v3KOEWJkMkvGBHfHKuf9FKYbWltHtUE6bAEO1orq0JayD8UNUKxdGkElXA7mkuIEexmBuieG9PJ2ow_uo05QCsqDvxlzOCMMIe7WdT7gmz4myiZ7lVuUcL1V2-Y1PJqWDyqDZbKNxd4X_CwW0RLOF1pw9S2URgybqHZFG0murNw

So the final JWT would be:

The image above was captured from the decoding tool injwt.io.

JWT is basically a Base64 encoded string with a signature attached to it. Havingthat signature component makes JWT a secure format for transferring data. It ispossible to simply encode data with Base64 and transfer that string. From theexample above, transferring the encoded payload"eyJ1c2VyIjoic2FtQGV4YW1wbGUuY29tIiwiZXhwIjoxNjc2MjYzNzYzfQ" can also get thedata to the other party. However, the party that receives the data have no wayof ensuring that the data was sent by a trusted source and that the data was nottampered with along the way.

Verifying JWT tokens

Anyone who has the secret key used to sign a JWT, can verify the integrity ofthe JWT. The steps involved in verifying the signature of a JSON Web Token (JWT)are:

1. Split the JWT into the encoded header, the encoded payload and the signature,using the dot (.) used to separate the three components.

2. Decode the encoded header and the encoded payload using Base64 to get theheader and payload JSON strings.

3. The application recreates the signature by signing the encoded header and theencoded payload using the signing algorithm in the header and the secret key.If the signature created in this step is the same as the signature in theJWT, the JWT is valid.

4. The application validates the claims in the payload if there are any.

It can be verified that the JWT was generated by your application and has notbeen tampered with, if the signature is valid. If the JWT header or payload wastampered with, the signature produced while verifying would be different fromthe one in the JWT.

Using JWT for the state parameter

We can generate a JWT token and pass that as the state parameter in the GitHubOAuth authorizing process. Here's how the process will be different when usingJWT:

1. The application requests authorization from GitHub. This time the stateparameter will be a JWT signed using a secure algorithm and a secret key. TheJWT need not be stored in memory.

2. GitHub gets the approval of the user.

3. GitHub sends a request to the application through the URI passed asredirect_uri in the first step. This request will contain the code andthe state parameters. Here, the state parameter would be a JWT.

4. Validate the JWT from the state parameter. If the JWT is invalid, drop theauthorization process immediately.

Advantages of using JWT for the state parameter

1. No Storage requirement - When using a random string for the stateparameter, that string has to be stored, so that it can be used later forverification. However, JWTs can be verified without storing them once theyare generated.

2. Ability to send additional data - The payload component of JWT can beused to send additional data such as user data, permissions data, etc.

3. Security - Using JWT can ensure that the OAuth process was initiated byyour application or a trusted source. In addition, JWT also ensures theintegrity of the data. This means that we can ensure that the payload in theJWT has not been tampered with.

4. Flexibility - The claims in the payload component of JWT can be usedfurther enhance the security of the JWT by implementing custom securitychecks based on the claims in the payload.

5. Standardization - JWT is a widely used format for transferring data.Using JWT would be beneficial when there are different applications andservices involved in the OAuth process.

References

1. OAuth 2.0 -https://www.digitalocean.com/community/tutorials/an-introduction-to-oauth-2

2. GitHub OAuth Authorization process -https://docs.github.com/en/developers/apps/building-oauth-apps/authorizing-oauth-apps

3. Phishing Attack -https://www.imperva.com/learn/application-security/phishing-attack-scam/

DNS management is a free service by Cloudflare. However, on first glance, itmight not appear that it's a free service. Once we add a site, then we see ascreen like this. Here, we need to remember to scroll down to see the freeoption.

Now let's see how we can map the DNS settings from Heroku to Cloudflare. We willlook at both a standard domain name and then we will take a look at a wildcarddomain name.

Standard domain name

We are hosting GitEmit using Heroku. We are lettingHeroku manages the SSL for this domain.

After setting up domains in Heroku, here is what we see.

In Cloudflare, we can set it up using two CNAMEs. It looks like this.

Wild card domain name

We are hosting NeetoChat application usingHeroku.

Since it's a wild card domain, we had togenerate the certificatesourselves.

After setting up domains in Heroku, here is what we see.

In Cloudflar,e we can set it up using three CNAMEs. It looks like this.

In such scenarios, the baseUrl will be different in both environments. Butcypress allows us to configure baseUrl only once in cypress.json . Then, howdo we configure baseUrl specific for each of the environments? In this blog,we will see how to configure cypress step by step so that we can run cypresstests in multiple environments.

1. Add environment specific configuration

To configure Cypress for different environments, we can follow the two simplesteps.

1. Create config folder in cypress.
2. Create separate files for each of the environments.

// cypress/config/cypress.development.json{  "baseUrl": "https://localhost:9006",  "env": {    "environment": "development"  },  "execTimeout": 18000,  "defaultCommandTimeout": 300000,  "requestTimeout": 10000,  "pageLoadTimeout": 30000,  "responseTimeout": 10000,  "viewportWidth": 1200,  "viewportHeight": 1200,  "videoUploadOnPasses": false,  "retries": {    "runMode": 1,    "openMode": 2  }}

// cypress/config/cypress.test.json{  "baseUrl": "https://test.example.com",  "env": {    "environment": "test"  },  "execTimeout": 300000,  "defaultCommandTimeout": 60000,  "requestTimeout": 20000,  "pageLoadTimeout": 60000,  "responseTimeout": 20000,  "viewportWidth": 1200,  "viewportHeight": 1200,  "videoUploadOnPasses": true,  "retries": {    "runMode": 2,    "openMode": 1  }}

// cypress/config/cypress.production.json{  "baseUrl": "https://live.example.com",  "env": {    "environment": "production"  },  "execTimeout": 300000,  "defaultCommandTimeout": 60000,  "requestTimeout": 20000,  "pageLoadTimeout": 60000,  "responseTimeout": 20000,  "viewportWidth": 1200,  "viewportHeight": 1200,  "videoUploadOnPasses": true,  "retries": {    "runMode": 2,    "openMode": 1  }}

2. Initialize config files

After adding config folder, we need to tell cypress about those config files. Wecan do that by updating plugins/index.js with following code. This file getsexecuted after we start the cypress server.

// cypress/plugins/index.jsconst fs = require("fs-extra");const path = require("path");const fetchConfigurationByFile = file => {  const pathOfConfigurationFile = `config/cypress.${file}.json`;  return (    file && fs.readJson(path.join(__dirname, "../", pathOfConfigurationFile))  );};module.exports = (on, config) => {  const environment = config.env.configFile || "development";  const configurationForEnvironment = fetchConfigurationByFile(environment);  return configurationForEnvironment || config;};

In the above code, Cypress loads the configuration file based on theenvironment. When we run Cypress we can pass environment variables. In thiscase, we need to pass configFile as an environment variable. If we don't passconfigFile, by default Cypress will consider development as the currentenvironment.

3. Setup scripts

Cypress can accept command-line arguments. We can set the environment by passingthese arguments in the cypress run or cypress open command e.g.cypress open --env configFile=test. This command looks lengthy. Also,sometimes we need to pass more command-line arguments along with configFile.We can create short and handy commands by configuring the package.json.

For example: yarn run cy:open:dev.

// cypress/package.json"cy:open:dev": "cypress open --env configFile=development","cy:open:dev:chrome": "cypress open --browser chrome --env configFile=development","cy:run:dev": "cypress run --env configFile=development","cy:open:staging": "cypress open --env configFile=test","cy:run:staging": "cypress run --env configFile=test",

4. Precedence of configuration

By default, we have cypress.json in Cypress where some config can be added.The precedence of configuration is higher for the files in the config folderthan the cypress.json. So if we have some key execTimeout defined in bothcypress.json and cypress.test.json. The value of execTimeout in thecypress.test.json will be considered.

Generally, the rule of thumb is that a common configuration for all theenvironments can be kept in the cypress.json. The environment-specific configcan be kept in config/ folder.

Conclusion

We saw that how we can configure the cypress in multiple environments. In thisway we can run Cypress for each environment without much hassle. This definitelyhelps us in running Cypress in both local as well as in the productionenvironment.

Cookies can be a very safe method for storage on the client side. They can beset as:

• HTTP only: cannot be accessed through client-side JavaScript, saving it fromany third party client-side scripts or web extensions.
• Secure: The web browser ensures that the cookies are set only on a securechannel.
• Signed: We can sign the content to make sure it isn't changed on the clientside.
• Same Site: Make sure that the cookie is sent only if the site matches yourdomain/subdomain (details)

Prerequisites

• Postgraphile - Generates an instant GraphQL API from a Postgres database
• Express - Minimalistic backend framework for NodeJS

Setup

We will start off with a base Express setup generated withexpress-generator.

const createError = require("http-errors");const express = require("express");const path = require("path");const cookieParser = require("cookie-parser");const logger = require("morgan");const app = express();require("dotenv").config();app.use(logger("dev"));app.use(express.json());app.use(express.urlencoded({ extended: false }));app.use(express.static(path.join(__dirname, "public")));// Use secret key to sign the cookies on creation and parsingapp.use(cookieParser(process.env.SECRET_KEY));// Catch 404 and forward to error handlerapp.use(function (req, res, next) {  next(createError(404));});// Error handlerapp.use(function (err, req, res) {  // Set locals, only providing error in development  res.locals.message = err.message;  res.locals.error = req.app.get("env") === "development" ? err : {};  // Render the error page  res.status(err.status || 500);  res.render("error");});module.exports = app;

FromPostgraphile's usage library pagefor adding Postgraphile to an express app:

app.use(  postgraphile(    process.env.DATABASE_URL || "postgres://user:pass@host:5432/dbname",    "public",    {      watchPg: true,      graphiql: true,      enhanceGraphiql: true,    }  ));

Now for the table setup. We need a private user_accounts table and a methodnamed authenticate_user that will return a JWT token of the form:

{  token: 'jwt_token_here',  username: '',  ...anyOtherDetails}

We will not be detailing table creation or authentication as there are many waysto go about it. But if you need help,Postgraphile security is thepage to rely on.

Adding the Plugin library

To attach a cookie to the request, we will use the @graphile/operation-hookslibrary which is open-sourcedon Github.

npm install @graphile/operation-hooks# ORyarn add @graphile/operation-hooks

To add the library to the app:

const { postgraphile, makePluginHook } = require("postgraphile");const pluginHook = makePluginHook([  require("@graphile/operation-hooks").default,  // Any more PostGraphile server plugins here]);app.use(  postgraphile(    process.env.DATABASE_URL || "postgres://user:pass@host:5432/dbname",    "public",    {      watchPg: true,      graphiql: true,      enhanceGraphiql: true,      pluginHook,      appendPlugins: [        // You will be adding the hooks here      ],    }  ));

Adding the Plugin

The plugin allows for two different types of hooks:

1. SQL Hooks
2. JavaScript Hooks

Since accessing cookies is a JavaScript operation, we will be concentrating onthe second type.

To hook the plugin into the build system, we can use the addOperationHookmethod.

module.exports = function OperationHookPlugin(builder) {  builder.hook("init", (_, build) => {    // Register our operation hook (passing it the build object):    // setAuthCookie is a function we will define later.    build.addOperationHook(useAuthCredentials(build));    // Graphile Engine hooks must always return their input or a derivative of    // it.    return _;  });};

If this is contained in a file named set-auth-cookie.js, then the plugin canbe added to the append plugins array as follows:

{  appendPlugins: [    require('./set-auth-cookie.js'),  ],}

Designing the hook

The function to be executed receives two arguments: build process and thecurrent fieldContext.

The fieldContext consists of fields that can be used to narrow down themutation or query that we want to target; e.g. if the hook is to run only onmutations, we can use the fieldContext.isRootMutation field.

const useAuthCredentials = build => fieldContext => {  const { isRootMutation } = fieldContext;  if (!isRootMutation) {    // No hook added here    return null;  }};

To direct the system on usage of the plugin, we have to return an object withbefore, after or error fields. Here is how these keywords can be used:

(comments are fromthe example repository)

return {  // An optional list of callbacks to call before the operation  before: [    // You may register more than one callback if you wish. They will be mixed in with the callbacks registered from other plugins and called in the order specified by their priority value.    {      // Priority is a number between 0 and 1000. If you're not sure where to put it, then 500 is a great starting point.      priority: 500,      // This function (which can be asynchronous) will be called before the operation. It will be passed a value that it must return verbatim. The only other valid return is `null` in which case an error will be thrown.      callback: logAttempt,    },  ],  // As `before`, except the callback is called after the operation and will be passed the result of the operation; you may return a derivative of the result.  after: [],  // As `before`; except the callback is called if an error occurs; it will be passed the error and must return either the error or a derivative of it.  error: [],};

Since we want our action to happen after we get result from the mutation, wewill add it to the after array.

const useAuthCredentials = build => fieldContext => {  const { isRootMutation, pgFieldIntrospection } = fieldContext;  if (!isRootMutation) {    // No hook added here    return null;  }  if (    !pgFieldIntrospection ||    // Name of the mutation is authenticateUser    pgFieldIntrospection.name !== "authenticateUser"  ) {    // narrowing the scope down to the mutation we want    return null;  }  return {    before: [],    after: [      {        priority: 1000,        callback: (result, args, context) => {          // The result is here, so we can access accessToken and username.          console.log(result);        },      },    ],    error: [],  };};

Since the functionality is inside the plugin hook, we do not have the expressresult to set the cookie .

But we do have an escape hatch with the third argument: context. Postgraphileallows us to pass functions or values into the context variable from thepostgraphile instance.

app.use(  postgraphile(process.env.DATABASE_URL, "public", {    async additionalGraphQLContextFromRequest(req, res) {      return {        // Function to set the cookie passed into the context object        setAuthCookie: function (authCreds) {          res.cookie("app_creds", authCreds, {            signed: true,            httpOnly: true,            secure: true,            // Check if you want to include SameSite cookies here, depending on your hosting.          });        },      };    },  }));

We can now set the cookie inside the plugin hook.

{  priority: 1000,  callback: (result, args, context) => {    // This function is passed from additionalGraphQLContextFromRequest as detailed in the snippet above    context.setAuthCookie(result);  }}

Reading from the Cookie

We have already added the cookieParser with SECRET_KEY, so express willparse the cookies for us.

But we probably want them to be accessible inside SQL functions forPostgraphile. That is how we can determine if the user is signed in or whattheir permissions are. To do that, Postgraphile provides a pgSettings object.

app.use(  postgraphile(process.env.DATABASE_URL, "public", {    pgSettings: async req => ({      user: req.signedCookies["app_creds"],    }),  }));

Inside an SQL function, the variables passed from settings can be accessed likethis:

current_setting('user')

That's all . We can store any details in cookies, retrieve them on the Expressend and use them inside Postgres functions for authentication or authorization.

Check out operation-hooks pluginfor more details.

First, let's see Authentication vs Authorization.

Authentication is determining whether a user is logged in or not.Authorization is then deciding what the users has permission to do or see.

Comparing the implementation of a blog application using Postgraphile vs REST

Suppose we have to build a blog application with the below schema.

Features of the blog application.

• Display published blogs with is_published = true to all users.

• Display unpublished blogs with is_published = false to its creatoronly.

REST Implementation

The REST implementation with JavaScript andsequelize canbe like below.

The client requests the blogs using an endpoint, it also attaches the accesstoken received from the authentication service.

const getBlogs = () =>  requestData({    endpoint: `/api/blogs`,    accessToken: "***",  });

The backend code in the server receives the request, finds the currentlogged in user from the access token, and requests the data based on the currentlogged in user from the database.

const userEmail = findEmail(accessToken);const blogs = await models.Blogs.findAll({  where: { [Op.or]: [{ creatorEmail: userEmail }, { isPublished: true }] },});res.send(blogs);

Here, the backend code finds the users email from the access token, thenrequests the database to give the list of blogs that have creatorEmail matchingto the current user's email or the field isPublished is true.

The database will return whatever data the server requests.

Similarly, for creating, editing, and deleting blogs, we can have differentend-points to handle the authorization logic in the backend code.

Postgraphile Implementation

The postgraphile implementation can be like below.

The client requests the blogs using a GraphQL query. It also attaches theaccess token received from the authentication service.

const data = requestQuery({ query: "allBlogs {         nodes {            content            creatorEmail            visiblityType           }        }" accessToken: '***'})

In the server, we configure Postgraphile to pass the user information to thedatabase.

export postgraphile(DATABASE_URL, schemaName, {  pgSettings: (req) => {     const userEmail = findEmail(accessToken);     return({         'current_user_email': userEmail     })  }})

We can pass a function as Postgraphiles pgSettingsproperty, whose return value will be accessible from the connected Postgresdatabase by calling the current_setting function.

In the database, the row-level security policies can be defined to controlthe data access.

Row-level security policiesare basically just SQL that either evaluates to true or false. If a policy iscreated and enabled for a table, that policy will be checked before doing anoperation on the table.

create policy blogs_policy_selecton public.blogs for select to usersUSING ( isPublished OR creator_email = current_setting('current_user_email'));ALTER TABLE blogs ENABLE ROW LEVEL SECURITY;

Here the policy named blogs_policy_select will be checked before selecting arow in the table public.blogs. A row will be selected only if theisPublished field is true or creator_email matches with the current user'semail.

Similarly, for creating, editing, and deleting blogs, we can have row levelsecurity policies for INSERT, UPDATE, and DELETE operations on the table.

Conclusion

The REST implementation does the authorization on the server level but thePostgraphile does it on the database level. Each implementation has its ownadvantages and disadvantages, which is a topic for another day.

Although, sorting based on associated tables columns or adding a custom sortcan be achieved via plugins. In this blog we will explore two such plugins.

Using pg-order-by-related plugin

pg-order-by-relatedplugin allows us to sort query result based on associated table's columns. Itdoes that by adding enums for all associated table's columns. Here's what weneed to do to use this plugin.

Installation

npm i @graphile-contrib/pg-order-by-related

Adding the plugin

const express = require("express");const { postgraphile } = require("postgraphile");const PgOrderByRelatedPlugin = require("@graphile-contrib/pg-order-by-related");const app = express();app.use(  postgraphile(process.env.DATABASE_URL, "public", {    appendPlugins: [PgOrderByRelatedPlugin],  }));

Using associated table column enum with orderBy argument

query getPostsSortedByUserId {  posts: postsList(orderBy: AUTHOR_BY_USER_ID__NAME_ASC) {    id    title    description    author: authorByUserId {      id      name    }  }}

pg-order-by-related plugin is useful only when we want to sort data based onfirst level association. If we want to apply orderBy on second level tablecolumns or so, we have to use makeAddPgTableOrderByPlugin.

Using makeAddPgTableOrderByPlugin

makeAddPgTableOrderByPluginallows us to add custom enums that are accessible on specified table's orderByargument. We can write our custom select queries using this plugin.

We will use a complex example to understand the use-case of custom orderByenum.

In our posts list query, we want posts to be sorted by author's address. Addresshas country, state and city columns. We want list to be sorted by country, stateand city in the same order.

Here's how we can achieve this using makeAddPgTableOrderByPlugin.

plugins/orderBy/orderByPostAuthorAddress.js

import { makeAddPgTableOrderByPlugin, orderByAscDesc } from "graphile-utils";export default makeAddPgTableOrderByPlugin(  "public",  "post",  ({ pgSql: sql }) => {    const author = sql.identifier(Symbol("author"));    const address = sql.identifier(Symbol("address"));    return orderByAscDesc(      "AUTHOR_BY_USER_ID__ADDRESS_ID__COUNTRY__STATE__CITY",      ({ queryBuilder }) => sql.fragment`(            SELECT              CONCAT(                ${address}.city,                ', ',                ${address}.state,                ', ',                ${address}.country              ) AS full_address            FROM public.user as ${author}            JOIN public.address ${address} ON ${author}.address_id = ${address}.id            WHERE ${author}.id = ${queryBuilder.getTableAlias()}.user_id            ORDER BY ${address}.country DESC, ${address}.state DESC, ${address}.city DESC            LIMIT 1          )`    );  });

Export all custom orderBy plugins

plugins/orderBy/index.js

export { default as orderByPostAuthorAddress } from "./orderByPostAuthorAddress";

Append custom orderBy plugins to postgraphile

const express = require("express");const { postgraphile } = require("postgraphile");import * as OrderByPlugins from "./plugins/orderby";const app = express();app.use(  postgraphile(process.env.DATABASE_URL, "public", {    appendPlugins: [...Object.values(OrderByPlugins)],  }));

Using custom enum with orderBy argument

query getPostsSortedByAddress {  posts: postsList(    orderBy: AUTHOR_BY_USER_ID__ADDRESS_ID__COUNTRY__STATE__CITY  ) {    id    title    description    author: authorByUserId {      id      name      address {        id        country        state        city      }    }  }}

Please head topg-order-by-relatedandmakeAddPgTableOrderByPluginpages for detailed documentation.

The ssl certificate in this example was bought from namecheap but the sameprocess would apply for other vendors too.

The video of the whole process is available here.

<iframewidth="100%"height="315"src="https://www.youtube.com/embed/A6URYtDWZhg"frameborder="0"allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"allowfullscreen

</iframe>

Script to generate keys

openssl req -new -newkey rsa:2048 -nodes -keyout server.key -out server.csr

When the prompt asks for Common name(full qualified host name) then enter*.yourdomainname.com. Since we are setting up a wild card certificate it'simportant that the common name starts with a *. Otherwise later we are goingto get an error.

Except the above mentioned question the answer to other questions do not matterat all. You can enter junk values and the SSL will work just fine.

Hit enter when a challenge password is requested.

Script to generate ssl bundle

$ cat __neetohelp_net.crt __neetohelp_net.ca-bundle > ssl-bundle.crt

Note that the order of the crt and bundle files matters when combining them.

Secondly, as shown in the video, we might have to split the combined line. Nowlet's examine the contents of the combined file.

$ cat ssl-bundle.crt

If we see a line like the one below:

-----END CERTIFICATE----------BEGIN CERTIFICATE-----

Then we need to split the line such that END and BEG align vertically likeso:

-----END CERTIFICATE----------BEGIN CERTIFICATE-----

We recently got a chance to see workspaces of our colleagues and everyone sharedphotos of environments they work in on Slack and it was fun seeing everyone'sdesk and the setup they have. We thought it would be fun to share a peek at thehome offices we have. Here we go.

Akhil Gautam

Amit Choudhary

Chimed Palden

Chirag Shah

Ershad Kunnakkadan

Mohit Natoo

Navaneeth PK

Neeraj Singh

Nitin Kalasannavar

Paras Bansal

Pranav Raj

Prathamesh Sonpatki

Rahul Mahale

Rishi Mohan

Shibin Madassery

Sony Mathew

Sunil Kumar

Tyler and Naiara

Unnikrishnan KP

Vishal Telangre

We achieved this to date without having any outbound marketing and salesstrategy.

• We have never sent a cold email.
• We have never sent a cold LinkedIn message.
• The only time we advertised was a period of two months when we tried Googleadvertisements, with no outcomes.
• We do not sponsor any podcast.
• We have not had a sales person.
• We have not had a marketing person.

We have kept our head down and have focused on what we do best, such asdesigning, developing, debugging, devops, and blogging.

This is what has worked out for us so far:

• We contribute to the community throughblog posts and open source.
• We sponsor community events like Rails Girls and Ruby Conf India.
• We sponsor many React and Ruby meetups.
• We focus on keeping our existing clients happy.

Over the years I have come across many people who aspire to be freelancers.While it is not for everyone, I encourage them to give freelancing a try.

The greatest hindrance I have seen is that they stress over sales and marketing,and as it should be. Being a freelancer means constant need to find your nextclient.

I'm not here to say what others ought to do. I'm here to say what has worked outfor BigBinary over the last 7 years.

While we plan to experiment with new forms of marketing, networking, and saleschannel as we grow, it is not the end-all-be-all for freelancers. Whilemarketing, networking, and sales may be effective for some, it was not how westarted BigBinary and may not be how you want to start as well.

For us at BigBinary, it has been writing blogs. When we come across apotentially intriguing blog topic, we save the topic by creating a Github issue.When we have downtime, we pick up a topic from our issues list. Its as simpleas that and has been our primary driver of growth thus far.

While you should experiment to find out what works best for you, you need tofind out what suits your personality. If you are good at teaching throughvideos, consider creating your own YouTube channel. If you contribute to opensource, try creating a blog about your efforts and learnings. If you are good atconcentrating on a niche technology, build your marketing and business aroundthat.

I can confidently say that majority of people I met and who want to befreelancer would do fine if they simply share what they are learning. Most ofthese people do technical work. Some of them already blog and others can blog. Ablog is a decent start nearly everybody will say. I'm saying that it is a goodend too.

If you do not want to do any other form of marketing then that's fine too. Justblogging will work out fine for you just like it has worked out fine for us atBigBinary.

Just because you are going to be a freelancer you dont have to change who youare. If you don't like sending cold emails then don't. If you do not likenetworking then thats alright as well. Write personal emails, dump corporatetalk, show compassion and be genuine.

So go on and do some freelancing. It would teach you a lot about softwaredevelopment, business, life, managing money, creating value and capturing value.It will be rough at times. And it would be hard at times. But it would also be aton of fun.

The "demo" database is actually a live client where sales team does demo.This ensures that the data that is copied is fresh and not stale.

We implemented a solution where we simply listed all the tables in namespace and used activerecord-importto copy the table data.We used activerecord-import gem to keep code agnostic of underlying database as we used different databases in development from production.Production is "SQL Server" and development database is "PostgreSQL".Why this project ended up having different database in development and in productionis worthy of a separate blog.

When we started using the above mentioned strategy thenwe quickly ran into a problem.Inserts for some tables were failing.

insert or update on table "dependent_table" violates foreign key constraint "fk_rails"Detail: Key (column)=(1) is not present in table "main_table".

The issue was we had foreign key constraints on some tables and "dependent" table was being processed before the "main" table.

So initially we thought of simply hard coding the sequence in which to process the tables. It means if any new table is added then we will have to update the service to include the newly added table. So we needed a way to identify the foreign key dependencies and determine the sequence to copy the tables at runtime. To resolve this issue, we thought of usingTopological Sorting.

Topological Sorting

To get started we need the list of dependencies of "main" and "dependent" tables.In Postgresql, this sql query fetches the table dependencies.

SELECT    tc.table_name AS dependent_table,    ccu.table_name AS main_tableFROM    information_schema.table_constraints AS tc    JOIN information_schema.key_column_usage AS kcu      ON tc.constraint_name = kcu.constraint_name      AND tc.table_schema = kcu.table_schema    JOIN information_schema.constraint_column_usage AS ccu      ON ccu.constraint_name = tc.constraint_name      AND ccu.table_schema = tc.table_schemaWHERE constraint_type = 'FOREIGN KEY'and (tc.table_name like 'namespace_%' or ccu.table_name like 'namespace_%');=> dependent_table  | main_table-----------------------------------   dependent_table1 | main_table1   dependent_table2 | main_table2

The above query fetches all the dependencies for only the tables have namespace or the tables we are interested in.The output of above query was [[dependent_table1, main_table1], [dependent_table2, main_table2]].

Ruby has a TSort module that for implementing topological sorts.So we needed to run the topological sort on the dependencies. So we inserted the dependencies into a hash and included the TSort functionality into the hash. Following is the way to include the TSort module into hash by subclassing the Hash.

require "tsort"class TsortableHash < Hash  include TSort  alias tsort_each_node each_key  def tsort_each_child(node, &block)    fetch(node).each(&block)  endend# Borrowed from https://www.viget.com/articles/dependency-sorting-in-ruby-with-tsort/

Then we simply added all the tables to dependency hash, as below

tables_to_sort = ["dependent_table1", "dependent_table2", "main_table1"]dependency_graph = tables_to_sort.inject(TsortableHash.new) {|hash, table| hash[table] = []; hash }table_dependency_map = fetch_table_dependencies_from_database=> [["dependent_table1", "main_table1"], ["dependent_table2", "main_table2"]]# Add missing tables to dependency graphtable_dependency_map.flatten.each {|table| dependency_graph[table] ||= [] }table_dependency_map.each {|constraint| dependency_graph[constraint[0]] << constraint[1] }dependency_graph.tsort=> ["main_table1", "dependent_table1", "main_table2", "dependent_table2"]

The output above, is the dependency resolved sequence of tables.

Topological sorting is pretty useful in situations where we need to resolve dependencies and Ruby provides a really helpful tool TSort to implement it without going into implementation details. Although I did spend time in understanding the underlying algorithm for fun.

Firefox extensions,similar to Chrome extensions,help us modifyandpersonalize our browsing experienceby adding new featuresto the existing sites.

Once we've publishedour extension to theMozilla's Add-on store(AMO),users who browse the AMOcan find the extensionand install it with one-click.But,if a user is already on our sitewhere a link is providedto the extension's AMO listing page,they would need tonavigate away from ourwebsite to the AMO,complete the install process,and then return back to our site.That is a bad user experience.

The inline installation enables us toinitiate the extension installationfrom our site.The extension can still be hosted on the AMObut userswould no longer have toleave our site to install it.

We had to try out a few suggested approachesbefore we got it working.

InstallTrigger

InstallTrigger (Link is not available)is an interfaceincluded in the Mozilla's Apps APIfor installing extensions.Using JavaScript,the install methodof InstallTrigger can be usedto start the download and installationof an extension (or anything packaged in a .xpi file)from a Web page.

A XPI(pronounced as "zippy")is similar to a zip file,which contains manifest file andthe install script for the extension.

So, let's try to install theGrammarly Extension for Firefox.To use it,we first need its .xpi file's location.Once we have published our extension on the AMO,we can navigate to the listings pagefor it and get the link for the .xpi.

For our present example,here's the listing page forGrammarly Extension.

Here,we can get the .xpi file's locationby right clicking on the+ Add to Firefox buttonand clicking on Copy Link Location.Note that the + Add to Firefox buttonwould only be visibleif we browse the link on a Firefox browser.Otherwise, it would be replaced bya Get Firefox Now button.

Once we have the URL,we can trigger the installation via JavaScripton our web page.

InstallTrigger.install({  "Name of the Extension": {    URL: "url pointing to the .xpi file's location on AMO",  },});

Pointing to the latest version of the Extension

When we used the URL in the above code,the .xpi file's URLwas specific to the extension's current version.If the extension has an update,the installed extensions for existing userswould be updated automatically.But the URL to the .xpi on our websitewould be pointing to the older version.Although the old link would still work,we would always want new users to downloadthe latest version.

To do that,we can either fetch the listing pagein the background and parse the HTMLto get the latest link.But that approach can break if the HTML changes.

Or we can query the Addons Services API,which returns the information for the extension in XML format.

For the Grammarly Extension, we first need its slug-id.We can get it by looking at its listing page's URL.From https://addons.mozilla.org/en-US/firefox/addon/grammarly-1/,we can note down the slug which is grammarly-1

Using this slug id, we can now get the extension details usinghttps://services.addons.mozilla.org/en-US/firefox/api/1.5/addon/grammarly-1.It returns the info for the Grammarly Extension.What we are particularly interested in is the value in the <install> node.That is what the desired value is for the latest version for our .xpi.

Let's see how we can implement the whole thing using React.

import axios from "axios";import cheerio from "cheerio";const FALLBACK_GRAMMARLY_EXTENSION_URL =  "https://addons.mozilla.org/firefox/downloads/file/1027073/grammarly_for_firefox-8.828.1757-an+fx.xpi";const URL_FOR_FETCHING_XPI = `https://services.addons.mozilla.org/en-US/firefox/api/1.5/addon/grammarly-1`;export default class InstallExtension extends Component {  state = {    grammarlyExtensionUrl: FALLBACK_GRAMMARLY_EXTENSION_URL,  };  componentWillMount() {    axios.get(URL_FOR_FETCHING_XPI).then((response) => {      const xml = response.data;      const $ = cheerio.load(xml);      const grammarlyExtensionUrl = $("addon install").text();      this.setState({ grammarlyExtensionUrl });    });  }  triggerInlineInstallation = (event) => {    InstallTrigger.install({      Grammarly: { URL: this.state.grammarlyExtensionUrl },    });  };  render() {    return (      <Button onClick={this.triggerInlineInstallation}>        Install Grammarly Extension      </Button>    );  }}

In the above code,we are using the npm packagesaxiosfor fetching the xml andcheeriofor parsing the xml.Also, we have set a fallback URL as the initial valuein case the fetching of the new URLfrom the xml response fails.

If we start puppeteer tests and do nothing in our laptop (just watch the testsbeing executed) then all the tests will pass.

However if we are doing our regular work in our laptop while tests are runningthen tests would fail randomly. This was quite puzzling.

Debugging such flaky tests is hard. We first suspected that the test casesthemselves needed more of implicit waits for element/text to be present/visibleon the DOM.

After some debugging using puppeteer protocol logs, it seemed like the browserwas performing certain actions very slowly or was waiting for the browser to beactive ( in view ) before performing those actions.

Chrome starting with version 57 introducedthrottling of background tabsfor improving performance and battery life. We execute one test per browsermeaning we didn't make use of multiple tabs. Also tests failed only when theuser was performing some other activities while the tests were executing inother background windows.Pages were hiddenonly when user switched tabs or minimized the browser window containing the tab.

After observing closely we noticed that the pages were making requests to theserver. The issue was the page was not painting if the page is not in view. Weadded flag --disable-background-timer-throttling but we did not notice anydifference.

After doing some searches we noticed the flag --disable-renderer-backgroundingwas being used inkarma-launcher.The comment states that it is specifically required on macOS. Here is thecoderesponsible for lowering the priority of the renderer when it is hidden.

But the new flag didn't help either.

While looking at all the available command line switches for chromium, westumbled upon --disable-backgrounding-occluded-windows. Chromium alsobackgrounds the renderer while the window is not visible to the user. It seemsfrom the comment that the flagkDisableBackgroundingOccludedWindowsForTestingis specifically added to avoid non-deterministic behavior during tests.

We have added following flags to chromium for running our integration suite andthis solved our problem.

const chromeArgs = [  "--disable-background-timer-throttling",  "--disable-backgrounding-occluded-windows",  "--disable-renderer-backgrounding",];

References

• Background tabs & offscreen frames
• Mac Window Occlusion API Use

Recently we started working with Elm in a couple of our projects -APISnapshot andAceHelp.

Tools like prettier and rubocop have really helped us take a load off our mindwith regards to formatting code. And one of the very first things we wanted tosort out when we started doing Elm was pretty printing our Elm code.

elm-format created byAaron VonderHaar formats Elm source code according toa standard set of rules based on the officialElm Style Guide.

Automatic code formatting

Let's setup git hook to automatically take care of code formatting. We canachieve this much like how we did it in our previouspost,using Husky andLint-staged.

Let's add Husky and lint-staged as dev dependencies to our project. And forcompleteness also include elm-format as a dev dependency.

npm install --save-dev husky lint-staged elm-format

Husky makes it real easy to create git hooks. Git hooks are scripts that areexecuted by git before or after an event. We will be using the pre-commit hookwhich is run after you do a git commit command but before you type in a commitmessage.

This way we can change and format files that's about to be committed by runningelm-format using Husky.

But there is one problem here. The changed files do not get added back to ourcommit.

This is where Lint-staged comes in. Lint-staged is built to run linters onstaged files. So instead of running elm-format on a pre-commit hook we would runlint-staged. And we can configure lint-staged such that elm-format is run on allstaged elm files.

We can also include Prettier to take care of all staged JavaScript files too.

Lets do this by editing our package.json file.

{  "scripts": {    "precommit": "lint-staged"  },  "lint-staged": {    "*.elm": ["elm-format --yes", "git add"],    "*.js": ["prettier --write", "git add"]  }}

All set and done!

Now whenever we do a git commit command, all our staged elm and JavaScriptfiles will get properly formatted before the commit goes in.

So, we thought about automating it with CircleCI, similar to continuousdeployment.

We are using the following configuration in circle.yml to continuously releasechrome extensions from the master branch.

workflows:  version: 2  main:    jobs:      - test:          filters:            branches:              ignore: []      - build:          requires:            - test          filters:            branches:              only: master      - publish:          requires:            - build          filters:            branches:              only: masterversion: 2jobs:  test:    docker:      - image: cibuilds/base:latest    steps:      - checkout      - run:          name: "Install Dependencies"          command: |            apk add --no-cache yarn            yarn      - run:          name: "Run Tests"          command: |            yarn run test  build:    docker:      - image: cibuilds/chrome-extension:latest    steps:      - checkout      - run:          name: "Install Dependencies"          command: |            apk add --no-cache yarn            apk add --no-cache zip            yarn      - run:          name: "Package Extension"          command: |            yarn run build            zip -r build.zip build      - persist_to_workspace:          root: /root/project          paths:            - build.zip  publish:    docker:      - image: cibuilds/chrome-extension:latest    environment:      - APP_ID: <APP_ID>    steps:      - attach_workspace:          at: /root/workspace      - run:          name: "Publish to the Google Chrome Store"          command: |            ACCESS_TOKEN=$(curl "https://accounts.google.com/o/oauth2/token" -d "client_id=${CLIENT_ID}&client_secret=${CLIENT_SECRET}&refresh_token=${REFRESH_TOKEN}&grant_type=refresh_token&redirect_uri=urn:ietf:wg:oauth:2.0:oob" | jq -r .access_token)            curl -H "Authorization: Bearer ${ACCESS_TOKEN}" -H "x-goog-api-version: 2" -X PUT -T /root/workspace/build.zip -v "https://www.googleapis.com/upload/chromewebstore/v1.1/items/${APP_ID}"            curl -H "Authorization: Bearer ${ACCESS_TOKEN}" -H "x-goog-api-version: 2" -H "Content-Length: 0" -X POST -v "https://www.googleapis.com/chromewebstore/v1.1/items/${APP_ID}/publish"

We have created three jobs named as test, build and publish and used thesejobs in our workflow to run tests, build the extension, and publish them to thechrome store, respectively. Every step requires the previous step to runsuccessfully.

Let's check each job one by one.

test:  docker:    - image: cibuilds/base:latest  steps:    - checkout    - run:        name: "Install Dependencies"        command: |          apk add --no-cache yarn          yarn    - run:        name: "Run Tests"        command: |          yarn run test

We use cibuilds docker image for this job.First, we do a checkout to the branch and then use yarn to installdependencies. Alternatively, we can use npm to install dependencies as well.Then, as the last step, we are use yarn run test to run tests. We can skipthis step if running tests is not needed.

build:  docker:    - image: cibuilds/chrome-extension:latest  steps:    - checkout    - run:        name: "Install Dependencies"        command: |          apk add --no-cache yarn          apk add --no-cache zip          yarn    - run:        name: "Package Extension"        command: |          yarn run build          zip -r build.zip build    - persist_to_workspace:        root: /root/project        paths:          - build.zip

For building chrome extensions, we use thechrome-extension image. Here, wealso first do a checkout and then, install dependencies using yarn. Note, we areinstall zip utility along with yarn because we need to zip our chrome extensionbefore publishing it in next step. Also, we are not generating version numberson our own. The version number will be picked from the manifest file. This stepassumes that we have a task named build in package.json to build our app.

The Chrome store rejects multiple uploads with the same version number. So, wehave to make sure to update the version number, which should be unique in themanifest file before this step.

In the last step, we use persist_to_workspace to make build.zip availablefor the next step, publishing.

publish:  docker:    - image: cibuilds/chrome-extension:latest  environment:    - APP_ID: <APP_ID>  steps:    - attach_workspace:        at: /root/workspace    - run:        name: "Publish to the Google Chrome Store"        command: |          ACCESS_TOKEN=$(curl "https://accounts.google.com/o/oauth2/token" -d "client_id=${CLIENT_ID}&client_secret=${CLIENT_SECRET}&refresh_token=${REFRESH_TOKEN}&grant_type=refresh_token&redirect_uri=urn:ietf:wg:oauth:2.0:oob" | jq -r .access_token)          curl -H "Authorization: Bearer ${ACCESS_TOKEN}" -H "x-goog-api-version: 2" -X PUT -T /root/workspace/build.zip -v "https://www.googleapis.com/upload/chromewebstore/v1.1/items/${APP_ID}"          curl -H "Authorization: Bearer ${ACCESS_TOKEN}" -H "x-goog-api-version: 2" -H "Content-Length: 0" -X POST -v "https://www.googleapis.com/chromewebstore/v1.1/items/${APP_ID}/publish"

For publishing of the chrome extension, we use thechrome-extension image.

We need APP_ID, CLIENT_ID, CLIENT_SECRET andREFRESH_TOKEN/ACCESS_TOKEN to publish our app to the chrome store.

APP_ID needs to be fetched fromGoogle Webstore Developer Dashboard.APP_ID is unique for each app whereas CLIENT_ID, CLIENT_SECRET andREFRESH_TOKEN/ACCESS_TOKEN can be used for multiple apps. Since APP_ID isgenerally public, we specify that in the yml file. CLIENT_ID, CLIENT_SECRETand REFRESH_TOKEN/ACCESS_TOKEN are stored as private environment variablesusing CircleCI UI. For cases when our app is unlisted in the chrome store, weneed to store APP_ID as a private environment variable.

CLIENT_ID and CLIENT_SECRET need to be fetched fromGoogle API console. There, we need toselect a project and then click on the credentials tab. If there is no project,we need to create one and then access the credentials tab.

REFRESH_TOKEN needs to be fetched from Google API. It also defines the scopeof access for Google APIs. We need to refer toGoogle OAuth2for obtaining the refresh token. We can use any language library.

In the first step of the publish job, we are attaching a workspace to accessbuild.zip, which was created previously. Now, by using all the required tokensobtained previously, we need to obtain an access token from Google OAuth API(Link is not available), which must be used to push the app to the chrome store.Then, we make a PUT request to the Chrome store API to push the app, and thenuse the same API again, to publish the app.

Uploading via API has one more advantage over manual upload. Manual uploadgenerally takes up to 1 hour to show the app in the chrome store. Whereasuploading using Google API generally reflects the app within 5-10 minutes,considering app does not go for a review by Google.

Apple announced iOS 12 at WWDC 2018 a few days back. Being honest, it was a bitdisappointing to see some of the most requested features not being there iniOS 12. Like users have been asking for Dark mode since before iOS 11, and theability to set default apps. Its more of an update focussed on performanceimprovements, usability, and compatibility. The fact that iOS 12 is alsoavailable for the iPhones Apple released 5 years back is a great effort fromApple to keep users happy. And unlike last couple of years, this time we decidedto calm our curiosities and installed iOS 12 beta 1 on our devices right awayafter Apple released it for developers. This blog is based on our experiencewith iOS 12 on iPhone 8 Plus.

Installing iOS 12 on your iPhone

First things first, make sure you have iPhone 5s or newer. And before gettingstarted, plug-in your phone to iTunes and take a full-backup in case your phonegets bricked while installing iOS 12, which is very unlikely.

Once done, download and installbeta profile{:target="_blank"} for iOS 12and then download and update from Software Update section just like youinstall a regular iOS update. Its a straightforward OTA update process whichyoure already familiar with.

Note: This beta profile is from a third-party developer and is not officiallyfrom Apple. Apple will officially release public beta in around a month.

Weve been running iOS 12 since last week now and here are our thoughts on theadditions and changes introduced in iOS 12.

iOS 12 is fast

The performance improvements are significant and definitely noticeable.Previously on iOS 11, while accessing spotlight by swiping down from homescreen, it used to lag. And not just that, sometimes keyboard didnt even usedto show up and we had to repeat the same action to make it work. But things arefaster and better in iOS 12. The keyboard gets up as soon as spotlight shows up.

Another thing that weve noticed is the multitasking shortcut for switching tothe last app by 3d touching on left wasn't that reliable in iOS 11, so much thatit was easy to ignore the feature altogether than to use it, but in iOS 12 thesame shortcut is very smooth. Although there are times when it still doesntwork well, but that's very rare.

Spotlight widgets load faster in iOS 12 than they used to in iOS 11. Apart fromthis, 3d touch feels pretty smooth too. Its good to see Apple squeezing out thepower to improve the already good performance in iOS.

Notifications

<section style="float: right; max-width: 280px; margin-left: 30px;"><img src="/blog/images/images_used_in_blog/2018/ios-12/ios-12-notifications.png" alt="Notifications in iOS 12"></section>

Notifications in iOS 11 are a mess, theres no grouping, theres no way toquickly control notifications, theres no way to set priorities. Apple has addednotifications grouping and better notification management in iOS 12, so nownotifications from the same app are grouped together and you can control howoften you want to get notifications from the apps right from the notification.

We think the implementation can be a whole lot better. For the notificationsthat are grouped, you get to see only the last notification from that app, abetter way wouldve been to show two or three notifications and cut the rest.Theres no Notification pinning or snoozing which could've been very usefulfeatures.

Screen time and App limits

Theres a new feature in iOS 12 called Screen Time which is more like bird's-eyeview of your phone usage. Theres a saying that you cant improve something thatyou cant measure. Screen Time is a feature thats going to be very useful foreveryone who wants to cut down time on Social apps or overall phone usage. Itshows you every little detail of how many apps you use and for how much time andat what times. Not only this, it also keeps track of how many times you pickedup your phone, and how many notifications you receive from the apps you have onyour phone.

Other useful sub-feature of Screen time is App limits, which allows you to setlimit on app usage based on app or category. So lets say you dont want to useWhatsApp for more than 30 mins a day, you can do that through App limits. Itworks for app categories including Games, Social Networking, Entertainment,Creativity, Productivity, Education, Reading & Reference, Health & Fitness. Soyou can limit it via category which works across apps. Plus, it syncs acrossyour other iOS devices, so you cant cheat that way.

Siri and Shortcuts app

<section style="float: right; max-width: 250px; margin-left: 30px;"><img src='/blog/images/images_used_in_blog/2018/ios-12/ios-12-siri-shortcuts.png' alt='Siri Shortcuts in iOS 12'></section>

In iOS 12, you can assign custom shortcuts to Siri to trigger specific actions,which not only works for System apps but also with third-party apps. So now ifyou want to send a specific message to someone on WhatsApp, you can assign acommand for that to Siri and you can trigger that action just from Siri usingthat command.

Apple has also introduced a new app in iOS 12 called Shortcuts. Shortcuts applets you group actions and run those actions quickly. Although Shortcuts appisnt there in iOS 12 beta 1 but we think its one of best addition in iOS 12.

Updated Photos app

Photos app now has new section called "For you", where it shows all the newAlbums, your best moments, share suggestions, photos and effect suggestions.This is more like the Assistant tab of Google Photos app. Also you can shareselected photos or albums with your friends right from the Photos app.

The Album tab in Photos app is redesigned for easier navigation. Also there's anew tab for Search which has been advanced, so you can now search photos usingtags like "surfing" and "vacation".

It's good to see Apple paying attention to up the Photos app but we still thinkGoogle Photos is a better option for average person considering it lets youstore photos in Cloud for free. Also photo organization in Google Photos is muchbetter than in new Photos app in iOS 12.

Enhanced Do Not Disturb Mode

Do Not Disturb in iOS 12 is enhanced to be more flexible. You can now enable DoNot Disturb mode to end automatically in an hour, or at night, or according toyour Calendar events, or even based on your location.

Not just that, Do Not Disturb has a new Bedtime mode enabling which will shutall your notifications during bed time and dim your display. And when you willwake up, it'll show you a welcome back message along with weather details on thelock screen.

Conclusion

There are other updates and under the hood improvements as well, like newMeasure app, redesigned iBooks app, tracking prevention, group FaceTime etc.Overall, we think its an okay update considering there are not as many bugs asthere should be according to Apple standards. The force touch in the keyboard todrag cursor doesnt work, Skype and some other apps crash, but for the mostpart, its good enough to be installed on your primary device.

One of the features of ApiSnapshot is support for Basic Authentication.

While we were rebuilding the whole application in Elm, we had to port the "AddBasic Authentication" feature. This feature can be accessed from the "More"drop-down on the right-hand side of the app and it lets user add username andpassword to the request.

Let's see how the Model of our Elm app looks.

type alias Model ={ request : Request.MainRequest.Model, response : Response.MainResponse.Model, route : Route}

Here is the Model in Request.MainRequest module.

type alias APISnapshotRequest ={ url : String, httpMethod : HttpMethod, requestParameters : RequestParameters, requestHeaders : RequestHeaders, username : Maybe String, password : Maybe String, requestBody : Maybe RequestBody}type alias Model ={ request : APISnapshotRequest, showErrors : Bool}

username and password fields are optional for the users so we kept them asMaybe types.

Note that API always responds with username and password whether userclicked to add Basic Authentication or not. The API would respond with anull for both username and password when a user tries to retrieve asnapshot for which user did not fill username and password.

Here is a sample API response.

{  "url": "http://dog.ceo/api/breed/affenpinscher/images/random",  "httpMethod": "GET",  "requestParams": {},  "requestHeaders": {},  "requestBody": null,  "username": "alanturning",  "password": "welcome",  "assertions": [],  "response": {    "response_headers": {      "age": "0",      "via": "1.1 varnish (Varnish/6.0), 1.1 varnish (Varnish/6.0)",      "date": "Thu, 03 May 2018 09:43:11 GMT",      "vary": "",      "cf_ray": "4151c826ac834704-EWR",      "server": "cloudflare"    },    "response_body": "{\"status\":\"success\",\"message\":\"https:\\/\\/images.dog.ceo\\/breeds\\/affenpinscher\\/n02110627_13221.jpg\"}",    "response_code": "200"  }}

Let's look at the view code which renders the data received from the API.

view : (Maybe String, Maybe String) -> Html Msgview usernameAndPassword =case usernameAndPassword of(Nothing, Nothing) -> text ""(Just username, Nothing) -> basicAuthenticationView username ""(Nothing, Just password) -> basicAuthenticationView "" password(Just username, Just password) -> basicAuthenticationView username passwordbasicAuthenticationView : String -> String -> Html MsgbasicAuthenticationView username password =[ div [ class "form-row" ][ input[ type_ "text", placeholder "Username", value username, onInput (UpdateUsername)][], input[ type_ "password", placeholder "Password", value password, onInput (UpdatePassword)][], a[ href "javascript:void(0)", onClick (RemoveBasicAuthentication)][ text "" ]]]

To get the desired view we apply following rules.

1. Check if both the values are string.
2. Check if either of the values is string.
3. Assume that both the values are null.

This works but we can do a better job of modelling it.

What's happening here is that we were trying to translate our API responsesdirectly to the Model . Let's try to club username and password together into anew type called BasicAuthentication.

In the model add a parameter called basicAuthentication which would be of typeMaybe BasicAuthentication. This way if user has opted to use basicauthentication fields then it is a Just BasicAuthentication and we can showthe input boxes. Otherwise it is Nothing and we show nothing!

Here is what the updated Model for Request.MainRequest would look like.

type alias BasicAuthentication ={ username : String, password : String}type alias APISnapshotRequest ={ url : String, httpMethod : HttpMethod, requestParameters : RequestParameters, requestHeaders : RequestHeaders, basicAuthentication : Maybe BasicAuthentication, requestBody : Maybe RequestBody}type alias Model ={ request : APISnapshotRequest, showErrors : Bool}

Elm compiler is complaining that we need to make changes to JSON decoding forAPISnapshotRequest type because of this change.

Before we fix that let's take a look at how JSON decoding is currently beingdone.

import Json.Decode as JDimport Json.Decode.Pipeline as JPdecodeAPISnapshotRequest : Response -> APISnapshotRequestdecodeAPISnapshotRequest hitResponse =letresult =JD.decodeString requestDecoder hitResponse.bodyincase result ofOk decodedValue ->decodedValue            Err err ->                emptyRequestrequestDecoder : JD.Decoder APISnapshotRequestrequestDecoder =JP.decode Request|> JP.optional "username" (JD.map Just JD.string) Nothing|> JP.optional "password" (JD.map Just JD.string) Nothing

Now we need to derive the state of the application from our API response .

Let's introduce a type called ReceivedAPISnapshotRequest which would be theshape of our old APISnapshotRequest with no basicAuthentication field. Andlet's update our requestDecoder function to return a Decoder of typeReceivedAPISnapshotRequest instead of APISnapshotRequest.

type alias ReceivedAPISnapshotRequest ={ url : String, httpMethod : HttpMethod, requestParameters : RequestParameters, requestHeaders : RequestHeaders, username : Maybe String, password : Maybe String, requestBody : Maybe RequestBody}requestDecoder : JD.Decoder ReceivedAPISnapshotRequest

We need to now move our earlier logic that checks to see if a user has opted touse the basic authentication fields or not from the view function to thedecodeAPISnapshotRequest function.

decodeAPISnapshotRequest : Response -> APISnapshotRequestdecodeAPISnapshotRequest hitResponse =letresult =JD.decodeString requestDecoder hitResponse.bodyincase result ofOk value ->letextractedCreds =( value.username, value.password )                    derivedBasicAuthentication =                        case extractedCreds of                            ( Nothing, Nothing ) ->                                Nothing                            ( Just receivedUsername, Nothing ) ->                                Just { username = receivedUsername, password = "" }                            ( Nothing, Just receivedPassword ) ->                                Just { username = "", password = receivedPassword }                            ( Just receivedUsername, Just receivedPassword ) ->                                Just { username = receivedUsername, password = receivedPassword }                in                    { url = value.url                    , httpMethod = value.httpMethod                    , requestParameters = value.requestParameters                    , requestHeaders = value.requestHeaders                    , basicAuthentication = derivedBasicAuthentication                    , requestBody = value.requestBody                    }            Err err ->                emptyRequest

We extract the username and password into extractedCreds as a Pair fromReceivedAPISnapshotRequest after decoding and construct ourAPISnapshotRequest from it.

And now we have a clean view function which just takes a BasicAuthenticationtype and returns us a Html Msg type.

view : BasicAuthentication -> Html Msgview b =[ div [ class "form-row" ][ input[ type_ "text", placeholder "Username", value b.username, onInput (UpdateUsername)][], input[ type_ "password", placeholder "Password", value b.password, onInput (UpdatePassword)][], a[ href "javascript:void(0)", onClick (RemoveBasicAuthentication)][ text "" ]]]

We now have a Model that better captures the business logic. And should wechange the logic of basic authentication parameter selection in the future, Wedo not have to worry about updating the logic in the view .

We built APISnapshot for two reasons.

We wanted to work with Elm.

We wanted to have a tool that is easy to use and that will help us capture whatresponse we are getting from the API in a format that is easy to share in githubissue, in slack or in an email. As a consulting company we work with variousteams around the world and during development phase either API is unstable orthey do not do what they should be doing.

We originally built this tool using React. Elm compileris quite strict and forced us to take into consideration all possibilities. Thislead us to notice a few bugs which were still present in React code. In this wayElm compiler helped us produce "correct" software by eliminating some of thebugs that we would have found later.

JSON encoding/decoding is a hard problem in Elm in general. In most of the caseswe know the shape of the API response we are going to get.

In the case of APISnapshot we do not know the shape of the JSON response we willget. Because of that it took us a bit longer to build the application. However,this forced us to really dig deep into JSON encoding/decoding issue in Elm andwe learned a lot.

We would like to thank all thecontributors tothe project. Special shout out goes to Jasim for thediscussion and the initial work on parsing the JSON file.

We like the combination of Elm and Ruby on Rails so much so that we are buildingAceHelp using the same technologies. AceHelp isopen source from day one.

Since puppeteer scripts execute so fast certain tests fail when they should bepassing. Debugging those tests can be a challenge.

Chrome devtools comes withfilmstrip feature. In"Performance" tab we can see screenshots of the site as they change over time.

We wanted puppeteer to generate similar filmstrip so that we can visuallyidentify the source of the problem.

It turns out that puppeteer makes it very easy. Here is the full code.

import puppeteer from "puppeteer";(async () => {  const browser = await puppeteer.launch({ headless: false });  const page = await browser.newPage();  await page.setViewport({ width: 1280, height: 1024 });  await page.tracing.start({ path: "trace.json", screenshots: true });  await page.goto("https://www.bigbinary.com");  await Promise.all([    page.waitForNavigation(),    page.click("#navbar > ul > li:nth-child(1) > a"),  ]);  await page.tracing.stop();  await browser.close();})();

If we execute this script then it will generate a file called trace.json. Thisfile has images embedded in it which are base64 encoded.

To see the filmstrip drag the trace.json file to "Performance" tab in theChrome devtool. Here is a quick video explaining this.

<iframewidth="560"height="315"src="https://www.youtube.com/embed/hZGTIyc3Xak"frameborder="0"allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture"allowfullscreen

</iframe>

An identity function is a very basic function that

• takes one argument
• returns the argument

f(x) = x;

This seems like the most useless function in the world. We never needed anyfunction like this while building any application. Then what's the big dealabout this identity function.

In this blog we will see how this identity concept is used in the real world.

For the implementation we will be using Ramda.js. Wepreviouslywrote abouthow we, at BigBinary, write JavaScript code using Ramda.js.

Again please note that in the following code R stands for Ramda and not forprogramming language R.

Example 1

Here is JavaScript code.

if (x) return x;return [];

Here is same code using Ramda.js.

R.ifElse(R.isNil, () => [], R.identity);

try it

Example 2

Here we will use identity as the return value in the default case.

R.cond([  [R.equals(0), R.always("0")],  [R.equals(10), R.always("10")],  [R.T, R.identity],]);

try it

Example 3

Get the unique items from the list.

R.uniqBy(R.identity, [1, 1, 2]);

try it

Example 4

Count occurrences of items in the list.

R.countBy(R.identity, ["a", "a", "b", "c", "c", "c"]);

try it

Example 5

Begin value from zero all the way to n-1.

R.times(R.identity, 5);

try it

Example 6

Filter truthy values.

R.filter(R.identity, [  { a: 1 },  false,  { b: 2 },  true,  "",  undefined,  null,  0,  {},  1,]);

try it

After researching we found that JWPlayer makes an AJAX request to load them3u8 file. To fix the issue, we neededto enable CORS and for that we needed to make changes to S3 and Cloudfrontconfigurations.

S3 configuration changes

We can configure CORS for the S3 bucket by allowing requests originating fromspecified hosts. As show in the image below we can find the CORS configurationoption in Permissions tab of the S3 bucket.Here is the officialdocumentation on configuring CORS for S3.

S3 bucket will now allow requests originating from the specified hosts.

Cloudfront configuration changes

Cloudfrontis a CDN service provided by AWS which uses edge locations to speed up thedelivery of static content. Cloudfront takes content from S3 buckets and cachesit at edge locations and delivers it to the end user. For enabling CORS we needto configure Cloudfront to allow forwarding of required headers.

We can configure the behavior of Cloudfront by clicking on CloudfrontDistribution's "Distribution Settings". Then from the "Behaviour" tab click on"Edit". Here we need to whitelist the headers that need to be forwarded. Selectthe "Origin" header to whitelist which is required for CORS, as shown in theimage below.

After attending the conference, I can say that Elm community is growing strong.The conference was attended by around 350 people and many were using Elm inproduction. More number of people wanted to try Elm in production.

There was a lot of enthusiasm about starting new Elm meetups. As a Ruby on Railsand React meetup organizer myself, I was genuinely interested in hearingexperiences of seasoned meetup organizers. In general Evan and Richard prefermeetup to be a place where people form small groups and hack on something ratherthan one person teaching the whole group something.

I liked all the talks. There was variety inthe topics and the speakers were all seasoned. Kudos to the organizers forputting up a great program. Below is a quick summary of my thoughts from theconference.

Keynote by Evan

Evan talked about the work he has been doing forthe upcoming release of Elm. He discussed the optimization work related to codesplitting, code generation and minification for speeding up building anddelivering single page apps using Elm. He made another interesting point that hechanged the codegen which generates the JS code from Elm code twice but nobodynoticed it. Things like this can give a huge opportunity to change and improveexisting designs which he has been doing for the upcoming release.

In the end he mentioned that his philosophy is not to rush things. It's betterto do things right than doing it now.

After the keynote, he encouraged people to talk to him about what they areworking on which was really nice.

Accessibility with Elm

Tessa talked about her work around addingaccessibility supportfor Elm apps. She talked about design decisions, prior art and some of thechallenges she faced while working on the library like working with tabs,interactive elements and images. There was a question at the end about whetherthis will be incorporated into Elm core but Evan mentioned that it might takesome time.

Putting the Elm Platform in the Browser

Luke, the creator ofEllie - a way to easily share your elm code withothers online - talked about how he started with Ellie. He talked about theproblems he had to face for implementing and sustaining Ellie through ads.During the talk, he also open sourced the code, so we can see it onGithub now.

Luke mentioned how he changed the architecture of Ellie from mostly running onthe server to running in the browser using service workers. He discussed futureplans about sustaining Ellie, building an Elm editor instead of usingCodemirror, getting rid of ads and making Ellie better for everyone.

The Importance of Ports

In other frameworks like PureScript andBuckleScript invoking native JavaScriptfunctions is easy. In Elm one has to use "Ports". Using Ports requires someextra stuff. In return we get more safety.

Murphy Randle presented a case where he wasusing too many ports which was resulting in fragmented code. He discussed howport is based on Actor Model andonce we get that then using port would be much easier. He also showed refactoredcode.

Murphy also runs Elm Town Podcast (Link is not available). Listen to episode 13to know more about Ports.

Keynote by Richard Feldman

Richard talked about his experiences inteaching beginners about Elm. He has taught Elm a lot. He has done an extensiveElm course on Front end masters. Heis currently writingElm in Action book.

He talked about finding motivation to teach using theSWBAT technique. It helped him in deciding theagenda and finding the direct path for teaching. He mentioned that in thebeginning being precise and detailed is not important. This resonated with me asthe most important thing for anyone who is getting started is getting startedwith the most basic things and then iterating over it again and again.

Parting thoughts

Elm community is small, tight, very friendly and warm. Lots of people are tryinga lot of cool things. Elm Slack came in thediscussions again and again as a good place to seek out help for beginners.

When I heard about Elm first, it was about good compiler errors and having runtime safety. However after attending the conference I am mighty impressed withthe Elm community.

Big props to Brian andLuke for organizing the conference!

All the videos from the conference are already gettinguploaded here.

Elm FAQ hasan answerto this question. However I could not fully understand the answer.

This is my attempt in explaining it.

What is type

In Elm everything has a type. Fire up elm-repl and you will see 4 is anumber and "hello" is a String.

> 44 : number> "hello""hello" : String

Let's assume that we are working with users records and we have followingattributes of those users.

• Name
• Age
• Status (Active or Inactive)

It's pretty clear that "Name" should be of type "String" and "Age" should be oftype "number".

Let's think about a moment what is the type of "Status". What is "Active" and"Inactive" in terms of type.

Active and Inactive are two valid values of Status. In other programminglanguages we might represent Status as an enum.

In Elm we need to create a new type. And that can be done as shown here.

type Status = Active | Inactive

Second thing we are doing is that we are stating that the valid values for thisnew type are Active and Inactive.

When I discussed this code with my team members they asked me to show where isActive and Inactive defined. Good question.

The simple answer is that they are not defined anywhere. They do not need to bedefined. What needs definition is the new type that is being created.

What makes understanding it a bit hard for people coming from Ruby, Java andsuch background is that these people (including me) are looking at Active andInactive as a class or a constant which is not the right way to look at.

Active and Inactive are the valid values for type Status.

> Active-- NAMING ERROR ----------Cannot find variable `Active`3|   Active     ^^^^^^

As you can see repl is not sure what Active is.

We can solve this by pasting following code in repl.

type Status = Active | Inactive

Now we can run the same code again. This time no error.

> ActiveActive : Repl.Status

What is type alias

Let's see a simple application which just prints name and age of a single user.

Hereis the code. I'm posting screenshot of the same below with certain parthighlighted.

As you can see { name : String, age : Int } is repeated at four differentplaces. In a bigger application it might get repeated more often.

This is what type alias does. It removes repetition. It removes verbosity.

As the name suggests this is just an alias. Note that type creates a new typewhereas type alias is literally saving keystrokes. type alias does notcreate a new type.

Now if you read the FAQ answer again then hopefully it will make morse sensenow.

Hereis modified code using type alias.

Why use type alias Username : String

While browsing Elm code in general, I came across following code.

type alias Username = String

Question is what does code like this buy us. All it does is that instead ofString I can now type Username.

First let's see how it might be used.

Let's assume that we have a function which returns Status of a user for thegiven username.

The function might have implementation as shown below.

getUserStatus username =  make_db_call_and_return_user_status

Now let's think about what the type annotation (rubyist think of it as methodsignature ) of function getUserStatus might look like.

It takes username as input and returns user record.

So the type annotation might look like

getUserStatus : String -> Status

This works. However the issue is that String is not expressive enough. It canbe made more expressive if the signature were

getUserStatus : Username -> Status

Now that we know about type alias all we need to do is

type alias Username = String

This makes code more expressive.

No recursion with type alias

An example of where we might need recursion is while designing commentingsystem. A comment can have sub-comments. However since type alias is just asubstitution and recursion does not work with it.

> type alias Comment = { message : String, responses : List Comment }This type alias is recursive, forming an infinite type!2| type alias Comment = { message : String, responses : List Comment }   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^When I expand a recursive type alias, it just keeps getting bigger and bigger.So dealiasing results in an infinitely large type! Try this instead:    type Comment        = Comment { message : String, responses : List Comment }This is kind of a subtle distinction. I suggested the naive fix, but you canoften do something a bit nicer. So I would recommend reading more at:<https://github.com/elm-lang/elm-compiler/blob/0.18.0/hints/recursive-alias.md>

Hint for Recursive Type Aliasesdiscusses this issue in greater detail and it also has solution to the problemof recursion.

Dual role of type alias as constructor and type

Let's say that we have following code.

type alias UserInfo =    { name : String, age : Int }

Now we can use UserInfo as a constructor to create records.

> type alias UserInfo = { name : String, age : Int }> sam = UserInfo "Sam" 24{ name = "Sam", age = 24 } : Repl.UserInfo

In the above case we used UserInfo as a constructor to create new userrecords. We did not use UserInfo as a type.

Now let's see another function.

type alias UserInfo =    { name : String, age : Int }getUserAge : UserInfo -> IntgetUserAge userinfo =    userinfo.age

In this case UserInfo is being used in type annotation as type and notas constructor.

Which one to use type or type alias

Both of them serve different purpose. Let's see an example.

Let's say that we have following code.

type alias UserInfo =    { name : String, age : Int }type alias Coach =    { name : String, age : Int, sports : String }

Now let's write a function that gets age of the given userinfo.

getUserAge : UserInfo -> IntgetUserAge UserInfo =    UserInfo.age

Now let's create two types of users.

sam = UserInfo "Sam" 24charlie = Coach "Charlie" 52 "Basketball"

Now let's try to get age of both of these people.

getUserAge samgetUserAge charlie

Here isthe complete versionif you want to run it.

Please note that elm-repldoes not support type annotationso you can't test this code in elm-repl.

The main point here is that since we used type alias, function getUserAgeworks for both UserInfo as well as Coach. It would be a stretch to say thatthis sounds like "duck typing in Elm" but it comes pretty close.

Yes Elm is statically typed language and it enforces type. However the pointhere is the type alias is not exactly a type.

So why did this code work.

It worked because of Elm's support forpattern matching forrecords.

As mentioned earlier type alias is just a shortcut for typing the verboseversion. So let's expand the type annotation of getUserAge.

If we were not using type alias UserInfo then it might have looked like asshown below.

getUserAge : { name : String, age : Int } -> Int

Here the argument is a record. Here isofficial guide on Records. While dealingwith records Elm looks at the argument and if that argument is a record and hasall the matching attributes then Elm will not complain because of its supportfor pattern matching.

Since Coach has both name and age attribute getUserAge charlie works.

You can test it by removing the attribute age from Coach and then you willsee that Compiler will complain.

In summary if we want strict type enforcement then we should go for type. Ifwe need something so that we do not need to type all the attributes all the timeand we want pattern matching then we should go for type alias.

It'shereand it looks like this.

length : String -> Int> String.length "Hello World"11 : Int

If we look at the similar feature in Ruby world then we getlength method.

length -> integer

Method signature in ruby's documentation and Elm's documentation is quitesimilar. Both return an integer.

In Elm's world method definitions are called "Type Annotations". Going forwardthat's what I'm going to use in this blog.

Now let's look at method definition of slice method in Ruby.

It looks like this.

slice(start, length) -> new_str or nilirb(main):006:0> "snakes on a plane!".slice(0,6)=> "snakes"

In Elm world it looks likethis.

slice : Int -> Int -> String -> String> String.slice  0  6 "snakes on a plane!""snakes" : String

Questions is what's up with all those arrows.

Producing a blog every 4 days consistently over 310 days takes persistence andtime - lots of it.

We needed to go through all the commits and then pick the ones which are worthwriting about and then write about it. Going into this I knew it would be a hardtask. Ruby on Rails is now a well crafted machine. In order to fully understandwhat's going on in the code base we need to spend sufficient time on it.

However I was surprised by the thing that turned to be the hardest - tellingstory of the code change.

Every commit has a story. There is a reason for it. The commit itself might beminor but that code change in itself does not tell the full story.

For example takethis commit.This commit is so simple that you might think it is not worth writing about.However in order to fully understand what it does we need to tell the full storywhich was captured inthis blog.

Or take the case ofRails 5 officially supports MariaDB . Theblog captures the full story and not just the code that changed.

Now you might say that I have cherry picked blog posts that favor my case. Solet's pick a blog which is simple.

You might wonder what could go wrong with a blog like this. As it turns out,plenty. That's because writing a blog also requires defining the boundary of theblog. Deciding what to include and what to leave out is hard. One gets a feelfor it only after writing it. And after having typed the words on screen,pruning is hard.

A good written article is simple writing. The problem with article which aresimple to readers is that - well it is simple. So it feels to readers thatwriting it must be simple. Nothing can be further from the truth. It takes a lotof hard work to produce anything simple. It's true in writing. And it's true inproducing software.

Coming back to the "Skipping Mailer" blog, it took quite a bit of back and forthto bring the blog to its essence. So yes the final output is quite short butthat does not mean that it took short amount of time to produce it.

Tell a story even if you have 10 seconds

John Lasseter was working as an animator at Disney in 1984. He was just firedfrom Disney for promoting computer animations at Disney. Lasseter joinsLucasfilm. Lucasfilm renamed itself to Pixar Graphics Group and sold itself toSteve Jobs for $5 million.

Lasseter was tasked with producing a short film that would show the power ofwhat computer animations could do so that Pixar Graphics Group can get someprojects like producing TV commercials with cartoon characters and earn somemoney. Lasseter needed to produce a short film for the upcoming computergraphics animation conference.

His initial idea was to have a short movie having a plotless character. Hepresented this idea to a conference in Brussels. There Belgian animator RaoulServais commented in slightly harsh tone that

No matter how short it is, it should have a beginning, a middle, and an end.Don't forget the story.

Lasseter complained that it's a pretty short movie and there might not be timeto present a story.

Raoul Servais replied

You can tell a story in ten seconds.

Lasseter started developing a character. He came up with the idea of Luxo Jr.

Here is final production ofLuxo Jr.

Luxo Jr. was a major hit at the conference. Crowd was on its feet in applauseeven before the two minutes film was over. Remember this is 1986 and ComputerAnimation was not much advanced at that time and this was the first movie evermade with the use of just computer graphics.

Lasseter later said that when audience was watching the movie they forgot thatthey were watching a computer animated film because the story took over them. Helearned the lesson that technology should enable better story telling andtechnology in itself divorced from story telling would not advance the cause ofPixar.

Later John Lasseter went on to produce hits like Toy Story, A bug's life, ToyStory 2, Cars, Cars 2, Monsters Inc, Finding Nemo and many more.

So you see even a great John Lasseter had to be reminded to tell a story.

Actual content over bullet points

Jeff Bezos is so focused on knowingthe full story that he banned usage of PowerPoint in internal meetings anddiscussions. As per him it is easy to hide behind bullet points in a PowerPointpresentation.

He insisted on writing the full story in word document and distribute it tomeeting attendees. The meetings starts with everyone head down reading thedocument.

He is also known for saying that if we are building a feature then we first needto know how it would be presented to the consumers when it is unveiled. We needto know the story we are going to tell them. Without the story we won't havefull picture of what we are going to build.

Learning to tell story is a journey

I'm glad that during the last 310 days 16 people contributed to the blog posts.The process of writing the posts at times was frustrating for a bunch of them.They had done the work of digging into the code and had posted their findings.Continuously getting feedback to edit the blog to build a nice coherent storywhere each paragraph is an extension of the previous paragraph is a downer. Somewere dismayed at why we are spending so much energy on a technical blog.

However in the end we all are happy that we underwent this exercise. We couldsee the initial draft of the blog and the final version and we all could see thedifference.

By no means we have mastered the art of storytelling. It's a long journey.However we believe we are on the right path. Hopefully in coming months andyears we at BigBinary would be able to bring to you more stories from changes inRails and other places.

$ netstat -nltProto Recv-Q Send-Q Local Address           Foreign Address         Statetcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTENtcp        0      0 127.0.0.1:11211         0.0.0.0:*               LISTENtcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTENtcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTENtcp        0      0 127.0.0.1:5432          0.0.0.0:*               LISTENtcp        0      0 127.0.0.1:3737          0.0.0.0:*               LISTENtcp6       0      0 :::22                   :::*                    LISTEN

As we can see above port 5432 is bound to 127.0.0.1. It means any attempt toconnect to the postgresql server from outside the machine will be refused. Wecan try hitting the port 5432 by using telnet.

$ telnet 107.170.11.79 5432Trying 107.170.11.79...telnet: connect to address 107.170.11.79: Connection refusedtelnet: Unable to connect to remote host

Configuring postgresql.conf

In order to fix this issue we need to find postgresql.conf. In differentsystems it is located at different place. I usually search for it.

$ find / -name "postgresql.conf"/var/lib/pgsql/9.4/data/postgresql.conf

Open postgresql.conf file and replace line

listen_addresses = 'localhost'

with

listen_addresses = '*'

Now restart postgresql server.

$ netstat -nltProto Recv-Q Send-Q Local Address           Foreign Address         Statetcp        0      0 127.0.0.1:11211         0.0.0.0:*               LISTENtcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTENtcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTENtcp        0      0 0.0.0.0:5432            0.0.0.0:*               LISTENtcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTENtcp        0      0 127.0.0.1:2812          0.0.0.0:*               LISTENtcp6       0      0 ::1:11211               :::*                    LISTENtcp6       0      0 :::22                   :::*                    LISTENtcp6       0      0 :::5432                 :::*                    LISTENtcp6       0      0 ::1:25                  :::*                    LISTEN

Here we can see that "Local Address" for port 5432 has changed to 0.0.0.0.

Configuring pg_hba.conf

Let's try to connect to remote postgresql server using "psql".

$ psql -h 107.170.158.89 -U postgrespsql: could not connect to server: Connection refusedIs the server running on host "107.170.158.89" and acceptingTCP/IP connections on port 5432?

In order to fix it, open pg_hba.conf and add following entry at the very end.

host    all             all              0.0.0.0/0                       md5host    all             all              ::/0                            md5

The second entry is for IPv6 network.

Do not get confused by "md5" option mentioned above. All it means is that apassword needs to be provided. If you want client to allow collection withoutproviding any password then change "md5" to "trust" and that will allowconnection unconditionally.

Restart postgresql server.

$ psql -h 107.170.158.89 -U postgresPassword for user postgres:psql (9.4.1, server 9.4.5)Type "help" for help.postgres=# \l

You should be able to see list of databases.

Now we are able to connect to postgresql server remotely.

Please note that in the real world you should be using extra layer of securityby using "iptables".

We added more clients. We increasedour team size. We wrote more blogs, spokeat more conferences, and made evenmore videos !

Here is the breakdown.

We love conferences

We presented at 10 conferences across 7 countries, on topics fromRails toReactJS.

• Garden City RubyConf, India
• RubyConf, Philippines
• RubyConf, India
• Reddot RubyConf, Singapore
• Deccan RubyConf, India
• Full Stack Fest, Spain
• RubyConf, Taiwan
• Rocky Mountain Ruby, Colorado
• RubyConf, Colombia

We are all in ReactJS

We at BigBinary adopted ReactJS pretty early. Early in the year we publishedseries of videos titledLearn ReactJS in stepswhich takes a "Hello World" app into a full TODO application using ReactJS inincremental steps.

Vipul and Prathamesh are currently authoring a book on ReactJS. Check it out atReactJS by Example- Building Modern Web Applications with React Book

We also started an ios app using React Native. It's coming along pretty good andsoon we should see it in app store.

We authored many blogs

We love sharing our experiences via blogs on various topics like ReactJS, Rails,React Native and Robot framework etc. Here are some of our blogs from 2015.

• How to obtain current time from a different timezone in Selenium IDE using javascript,Prabhakar, Jan 2015
• Author information in jekyll blog,Neeraj, Jan 2015
• Phone verification using SMS via Twilio,Santosh, Jan 2015
• Blue border around JWPLAYER video,Prathamesh, Feb 2015
• Gotcha with after_commit callback in Rails,Prathamesh, March 2015
• Voice based phone verification using twilio,Santosh, March 2015
• Verifying PubSub Services from Rails using Redis,Vipul, May 2015
• Using ReactJS with Rails Action Cable,Vipul, July 2015
• How to test React Native App on a real iPhone,Chirag, Aug 2015
• Optimize JavaScript code using BabelJS,Prathamesh, Aug 2015
• Configuring Pycharm IDE to run a Robot Framework test suite or a single test script,Prabhakar, Oct 2015
• Migrating rails app from postgresql to sql server,Rohit, Oct 2015
• Getting around Apple iTunesConnect account activation issue,Prathamesh, Oct 2015
• Rails 5 allows setting custom HTTP Headers for assets,Vipul, Oct 2015
• Using Stripe API in React Native with fetch,Chirag, Nov 2015
• How constant lookup and resolution works in Ruby on Rails,Mohit, Nov 2015
• Explicitly ssh into vagrant machine,Neeraj, Dec 2015
• ApplicationRecord in Rails 5, Prathamesh, Dec2015

Video Summary

• Learn Ruby on Rails
  • Use uuid, X-Request-Id and tagged logging to debug rails application.
  • Rails development using vagrant.
  • Using ES6 in Rails application.
• Learn ReactJS in step
• Keep up with ReactJS
• Learn JavaScript
  • Refactor JavaScript code using module pattern
  • A review of tools to test ES6
• Learn Selenium

Open Source

Apart from our team members contributing to various OpenSource projects, we alsosupport some projects from our team. This year, we added and helped buildfollowing projects-

• Wheel : Wheel is our Rails template fornew Ruby on Rails projects, with sane defaults and setups for differentenvironments, and common functionalities like image uploaders, debugging, etc.
• Mail Interceptor :Interception, Forwarding emails in Ruby on Rails application
• Handy : A collection handy tools andRails tasks for your Project.
• Learn ReactJS in Steps: Collection of examples fromLearn ReactJS in stepvideo series.
• Doctsplit Chef(Link is not available) : Check cookboxfor docsplit ruby gem
• Fixtures Dumper(Link is not available) : Dump yourRails data to fixtures easily to a database to populate data.

Community Engagement

Along with speaking at various conferences, we also helped organize, our Funedition of Pune's regional RubyConf,DeccanRubyConf, and supported other IndianConferences, including RubyConfIndia,GardenCity RubyConf.

We also help Pune's local ruby meetup,which had a splendid engagement this year.

Overall we are super excited about what we accomplished in year 2015. We arelooking forward to an exciting year of 2016!

vagarnt ssh

While working with chef I needed to explicitly ssh into the vagrant machine. Ittook me sometime to figure it out.

The key is command vagrant ssh-config. The output might look like this.

$ vagrant ssh-configHost vmachine  HostName 127.0.0.1  User vagrant  Port 2222  UserKnownHostsFile /dev/null  StrictHostKeyChecking no  PasswordAuthentication no  IdentityFile /Users/nsingh/code/vagrant-machine/.vagrant/machines/vmachine/virtualbox/private_key  IdentitiesOnly yes  LogLevel FATAL  ForwardAgent yes

Open ~/.ssh/config and paste the output at the end of the file and save thefile.

Now I can ssh into vagrant machine using ssh command as shown below.

ssh vmachine

If you are wondering from where the name vmachine came then this is the name Ihad given tomy vagrant machine.

Configuration to run a single test script

Running a single test script

Configuration to run a particular test suite

Running a particular test suite

This is because of the CSS class jwplayer-tab-focus.

The blue borderaround currently selected videoallows to identifywhich instance of JWPlayer is in focus.

But with a single JWPlayer instance, it can be annoying.

To remove this blue border,we can override the default JWPlayer CSS as follows.

.jw-tab-focus:focus {  outline: none;}

To keep all the overridden CSS in once place,we can add this change in a separate file such as jwplayer_overrides.css.

Short summary

In year 2014we presented atfollowing 8 conferencesacross 6 countries.

• RubyConf Goa, India
• RubyConf Philippines
• RedDotRubyConf Singapore
• DeccanRubyConf Pune, India
• Madion+Ruby Wisconsin, USA
• RubyConf Brazil
• RubyKaigi Tokyo, Japan
• Golden Gate Ruby Conference San Francisco, USA

Rails and Ruby Conferences

<iframe style="min-height: 500px; width: 100%;"src="https://www.mapquest.com/embed?hk=1x7D17Q"marginwidth="0"marginheight="0"frameborder="0"scrolling="no"></iframe>

At the start of our travel, we visited RubyConfIndia 2014.The conference took place at an amazing beach resort in Goa. The two days of conferencewere full of fun and interactions with the best ruby people around India. During theconference our team announced launch of Ruby India to helpspread ideas and experiments from the Ruby Indian Community, as well as highlightcontent from people.

Soon after, I visited Philippines, to conduct a workshop on "Contributing to Rails" atRubyConf Philippines. It was amazing to meet the growingPhilippines community. I was happy to spend time with some amazing Rubyists, likeNick Sutterer, Zachary Scott,Konstantin Hasse,Koichi Sasada San,PJ Hagerty,Andre Arko and so on.

After that I and Prathamesh went to Singapore to speak atRedDotRubyConf. We spoke on Arel andActiveRecord. "RedDorRubyConf" was our first joint talk together at a conference.Again we met a lot of awesome people like Piotr Solnica,Grzegorz Witek,Yinquan Teo,Sayanee Basu,Chinmay Pendharkarand Winston Teo Yong Wei.We also visited Marina Sand Bay and Sentosa Island.

Back in Pune, we hosted the first ever DeccanRubyConf.Our team was busy working on tasks right from building the website, inviting speakers,planning, and other arrangements. The conference had good talks and some really useful workshops.it was a fun one day conference, with attendance of over 170+ people.

The conference also saw our team announce the launch of RubyIndia Podcast,which does regular podcast interviews with notable people from the Ruby Community andIndian Community.

Prathamesh and I, then left on around a one and a half monthtravel, to attend and speak at multiple conferences.

We started with Madison+Ruby, in Madison, WI. After severalmissed flights, and a storm, we visited our first US conference after a travel of 48hours. MadisonRuby was a conference like no other. Several topics touched the humaneside of Ruby and the community. We spoke on 'Building an own ORM using ARel'. Set in thecultural town of Madison, we immensely enjoyed the cheese-curds, farmers markets and game nightarranged by the Conf team. A huge thanks to Jim andJennifer Remsik,for hosting such an amazing event. Thanks also to Scott Ruttencutterfor giving us space to work from his office and giving us a tour of the state capital.

We then visited Sao Paulo, Brazil for RubyConf Brazil and presented a talkon Building an ORM using ARel. It was a pleasure to meet Fabio Akitaand the CodeMiner team. We made friends with Celso Fernandesand Paulowho were kind enough to help us around, since in Brazil primarilyportuguese is spoken. We also met Rafael Franca andCarlos Antonio da Silvawho have helped us a lot with Rails issue tracker.

Next Prathamesh headed to RubyKaigi, being held in Tokyo, Japan.He presented onFixtures in Rails.He met Matz, creator of Ruby on his first day in Japan. Mostly all thecore Ruby contributors attended RubyKaigi. He got to interact withKoichi Sasada San, Jonan Scheffler,Akira Matsuda,Godfrey Chan,Richard Schneeman and lot of awesome Rubyists.He also met with his JRuby Core Tom Enebo for the first time.Thanks to Yasuo Honda, Michael Reinsch for helping with Japanese food.

From Brazil, I first visited Miami, and was happy to visit The Lab Miami,WynCode and interact with Rubyists from Miami.

Before heading to San Francisco, for GoGaRuco, I was able to make a quick stop in Boston and visitAlterConf Boston. The theme of the conference was arounddiversity in tech and gaming industry.

My latest conference was in the amazing city on San Francisco. I presented about 'Building an ORM',at GoGaRuCo, which incidentally was the last ever GoGaRuCo. The conferencetaking place in San Francisco, saw an amazing turnout of crowd. I was able to interact withSarah Allen, Yehuda Katz,Pat Allen, Sarah Mei,Ryan Davis. I spent most of the time along withNathan Long, Randy Coulman,and Nathan's friend Michael Gundlach,who is the creator of popularplugin Adblock.I also ran into Chris Eppstein, creator of compass.All around it was one of the most amazing interactions I had in a conference.

2014, was an amazing year for our team. Together we presented or were part of 8 conferences,launched RubyIndia Newsletter as well as the RubyIndia Podcast, started with 6 new videoseries on topics from ReactJS toRubymotion toSelenium, published numerous blogs,and contributed to a number of OpenSource projects.

2015, starts with our team presenting at GardenCityRuby Conf.We hope to get more such chances to interact and help out the community. Onwards to a new year!

Here are the specific steps I took which brought the running time to under 15minutes.

Set to run at the maximum speed

<table><tr><td> Command </td><td> Target </td><td> Value </td></tr><tr><td>setSpeed </td><td>0 </td><td></td></tr></table>

setSpeed command takes Target value in milliseconds. By setting the value tozero, I set the speed to maximum and the tests indeed ran fast. However, now Ihad lots of tests failing which were previously passing.

What happened.

In our tests real firefox browser is fired up and real elements are clicked. Theapplication does make round trip to the rails server hosted on heroku.

By setting the selenium tests to the maximum speed the selenium tests startedasserting for elements on the page even before the pages were fully loaded bythe browser.

I needed sets of instructions using which I could tell selenium how long to waitfor before asserting for elements.

Selenium provides a wonderful suite of commands which helped me fine tune thetest run. Here I'm discussing some of those commands.

waitForVisible

This command is used to tell selenium to wait until the specified element isvisible on the page.

In the below mentioned case, the Selenium IDE will wait until the elementcss=#text-a is visible on the page.

<table><tr><td> Command</td><td> Target</td><td> Value</td></tr><tr><td>waitForVisible</td><td>css=#text-a</td><td> </td></tr></table>

waitForText

This command is used to tell selenium to wait until a particular text is visiblein the specified element.

In the case mentioned below, Selenium IDE will wait until the text violet isdisplayed in the element css=#text-a.

<table><tr><td> Command</td><td> Target</td><td> Value</td></tr><tr><td>waitForText</td><td>css=#text-a</td><td>violet</td></tr></table>

<br />

The difference between waitForVisible and waitForText is thatwaitForVisible waits until the specified element is visible on the pagewhile waitForText waits until a particular text is visible in the specifiedelement on the page.

waitForElementPresent

This command is used to tell Selenium to wait until the specified element isdisplayed on the page.

In the below mentioned case, the Selenium IDE will wait until the elementcss=a.button is displayed on the page.

<table><tr><td> Command</td><td> Target</td><td> Value</td></tr><tr><td>waitForElementPresent</td><td>css=a.button</td><td></td></tr></table>

waitForVisible and waitForElementPresent seem very similar. It seems both ofthese commands do the same thing. There is a subtle difference though.

waitForVisible waits until the specified element is visible. Visibility of anelement is manipulated by the settings of CSS properties. For example usingdisplay none; one can make an element not be visible at all.

In contrast the command waitForElementPresent waits until the specifiedelement is present on the page in the form of html markup. This command does notgive consideration to css settings.

refreshAndWait

This command is used to tell Selenium to wait until the page is refreshed andthe targeted element is displayed on the web page.

In the example mentioned below, the Selenium IDE will wait until the page isrefreshed and the targeted element css=span.button is displayed on the page.

<table><tr><td> Command</td><td> Target</td><td> Value</td></tr><tr><td>refreshAndWait</td><td>css=span.button</td><td></td></tr></table>

clickAndWait

This command is used to tell selenium to wait until a particular button isclicked for submitting the form and the page starts reloading. The subsequentcommands are paused until, the page is reloaded after the element is clicked onthe page.

In the case mentioned below, Selenium IDE will wait until the page is reloadedafter the specified element css=input#edit is clicked.

<table><tr><td> Command</td><td> Target</td><td> Value</td></tr><tr><td>clickAndWait</td><td>css=input#edit</td><td></td></tr></table>

Selenium IDE commands used above and more are available atSelenium documentation.

Time to time we rebase the branch. And after rebasing we need to do force pushto send the changes to github. And once in a while someone force pushes intomaster by mistake. We recommend to set push.default to currentto avoid such issues but still sometimes force push does happen in master.

In order to prevent such mistakes in future we are usingpre-push hook.This is a small ruby program which runs before any git push command. If youare force pushing to master then it will reject the push like this.

*************************************************************************Your attempt to FORCE PUSH to MASTER has been rejected.If you still want to FORCE PUSH then you need to ignore the pre_push git hook by executing following command.git push master --force --no-verify*************************************************************************

Requirements

pre-push hook wasadded to gitin version 1.8.2. So you need git 1.8.2 or higher. You can easily upgrade git byexecuting brew upgrade git .

$ git --versiongit version 1.8.2.3

Setting up hooks

In order for these hooks to kick in they need to be setup.

First step is to clone the repo toyour local machine. Now open ~/.gitconfig and add following line.

[init]  templatedir= /Users/neeraj/code/tiny_scripts/git-hooks

Change the value /Users/neeraj/code/tiny_scripts/git-hooks to match with thedirectory of your machine.

Making existing repositories aware of this hook

Now pre-push hook is setup. Any new repository that you clone will have thefeature of not being able to force push to master.

But existing repositories do not know about this git-hook. To make existingrepositories aware of this hook execute following command on all repositories.

$ git initReinitialized existing Git repository in /Users/nsingh/dev/projects/streetcommerce/.git/

Now if you look into the .git/hooks directory of your project you should see afile called pre-push.

$ ls .git/hooks/pre-push.git/hooks/pre-push

It means this project is all set with pre-push hook.

New repositories

When you clone a repository then git init is invoked automatically and youwill get pre-push already copied for you. So you are all set for all futurerepositories too.

How to ignore pre-push hook

To ignore pre-push hook all you need to do is

# Use following command to ignore pre-push check and to force update master.git push master --force --no-verify

The hook is here.

git clone git@github.com:neerajsingh0101/rails.git

Now cd rails and execute git remote -v . This is what I see.

origin git@github.com:neerajsingh0101/rails.git (fetch)origin git@github.com:neerajsingh0101/rails.git (push)

Now I will add upstream remote by executing following command.

git remote add upstream git@github.com/rails/rails.git

After having done that when I execute git remote -v then I see

origin git@github.com:neerajsingh0101/rails.git (fetch)origin git@github.com:neerajsingh0101/rails.git (push)upstream git://github.com/rails/rails.git (fetch)upstream git://github.com/rails/rails.git (push)

Now I want to make some changes to the code. After all this is why I forked therepo.

Let's say that I want to add exception handling to the forked code I havelocally. Then I create a branch called exception-handling and make all yourchanges in this branch. The key here is to not to make any changes to masterbranch. I try to keep master of my forked repository in sync with the masterof the original repository where I forked it.

So now let's create a branch and I will put in all my changes there.

git checkout -b exception-handling

In the Gemfile I will use this code like this

gem 'rails', github: 'neerajsingh0101/rails', branch: 'exception-handling'

A month has passed. In the meantime rails master has tons of changes. I wantthose changes in my exception-handling branch. In order to achieve that firstI need to bring my local master up-to-date with rails master.

I need to switch to master branch and then I need to execute following commands.

git checkout mastergit fetch upstreamgit rebase upstream/mastergit push

Now the master of forked repository is in-sync with the master of rails/rails.Now that master is up-to-date I need to pull in the changes in master in myexception-handling branch.

git checkout exception-handlinggit rebase mastergit push -f

Now my branch exception-handling has my fix on top of rails master.