The Basics of DNS Records

A Domain Name System (DNS) is a hierarchical and decentralized naming system forcomputers and servers connected to the Internet. It essentially translateshuman-readable domain names into machine-readable IP addresses.

What is a Domain Name?

A Domain Name is a unique, human-friendly label used to identify one or more IPaddresses. It's what people type into a web browser to visit a website. Forexample, google.com and wikipedia.org are domain names.

Purchasing a New Domain Name

You don't buy a domain name outright. Technicall,y you register the right to useit for a specified period (typically 1 to 10 years) from a Domain NameRegistrar. These are called "registrars" because you are registering the rightto use the domain name.

Popular Domain Name Registrars are GoDaddy, Namecheap, Cloudflare and Porkbun.

DNS Records: Types and Uses

DNS Records provide information about a domain, primarily mapping hostnames toIP addresses. There are many different types of DNS records. We would be lookingat the important ones.

A Record

"A record" stands for "Address record". It maps a domain name (like google.com)to an IPv4 address (like 93.184.216.34).

AAAA Record

This does exactly the same thing as A record but for the IPv6 address. Four Assignifies a bigger address range.

CNAME Record

"CNAME record" stands for "Canonical Name Record".

Instead of pointing directly to an IP address like an A record does, a CNAMEpoints one hostname to another hostname.

An example of CNAME record could be: blog.bigbinary.com ->bigbinary.github.io.

What CNAME is saying here is that if you are visiting blog.bigbinary.com, thenthis record doesn't have the IP address information. If you want an IP address,then ask for that info from bigbinary.github.io. Remember that IP addressesare provided by A and AAAA records.

Another way of looking at it is that an A record provides the IP address,while CNAME provides the address of the next machine to which we can ask for theIP address.

Please note that it is quite possible that the next machine might have anotherCNAME and that in turn might have another CNAME.

The main point is that CNAME keeps the record of the next machine to ask for. Inthe end, some machine has to have an A or AAAA record so that a domain is mappedto an IP address.

MX Record

"MX Record" stands for "Mail Exchanger Record".

It specifies the mail servers responsible for accepting email messages on behalfof of a domain.

TXT Record

A TXT record (Text record) lets a domain owner store arbitrary text data in DNS.Other services then look up that text to verify ownership, check email policies,etc. Think of it like a sticky note that the domain owner can add for others toread.

Here are some use cases of the TXT record:

Adding SPF records

bigbinary.com. IN TXT "v=spf1 ip4:198.51.100.10 include:_spf.google.com -all"

Domain verification

Here is an example of site verification by Google Site.

bigbinary.com. IN TXT "google-site-verification=asduiashd9812h98asdh"

Adding DMARC records

_dmarc.bigbinary.com. IN TXT "v=DMARC1; p=reject; rua=mailto:dmarc@example.com"

NS Record

NS stands for "Name servers".

Let's say that a user types in the URL https://blog.neeto.com. The job of theDNS system is to ultimately find an IP address to which the request can be sent.Now, let's think about how to go about searching for the IP address.

We need to ask some machine for more information about blog.neeto.com. Thatinformation could be a CNAME record, an A record, or a AAAA record. But theproblem is who should be asked. From where we start the journey.

That's where the NS record comes into the picture. Remember, each domain isregistered with a registrar. So that's where the DNS system starts the journey.First, DNS will ask the registrar of the domain, "Hey, can you give me the Nameservers to whom I can start asking questions?"

Let's take a practical case and execute the following command on the terminal:

whois neeto.com

We'll get a lot of data, but we want to concentrate on the following six lines:

Domain Name: NEETO.COMRegistrar WHOIS Server: whois.cloudflare.comRegistrar: Cloudflare, Inc.Registrar URL: https://www.cloudflare.comName Server: BARBARA.NS.CLOUDFLARE.COMName Server: YEW.NS.CLOUDFLARE.COM

We can see that the Registrar for neeto.com is "Cloudflare". We also see thatthe Name servers are two servers, which are also clouflare.com. The people whodesigned name servers dictated that there should always be at least two nameservers. In this way, if one of them goes down for any reason, the overallsystem is still working.

Taking our example of blog.neeto.com, if we need a CNAME record, an A record,or a AAAA record, then we need to start asking those questions to these two nameservers.

Now let's do the same experiment with another domain. This time, we will choosehttps://bigbinary.com.

whois bigbinary.com

This time, we will get the following lines.

Domain Name: BIGBINARY.COMRegistrar: GoDaddy.com, LLCRegistrar WHOIS Server: whois.godaddy.comRegistrar URL: http://www.godaddy.comName Server: BARBARA.NS.CLOUDFLARE.COMName Server: YEW.NS.CLOUDFLARE.COM

Notice that this time the registrar is GoDaddy, but the Name servers arecloudflare.com. This is because bigbinary.com is registered with GoDaddy.When a domain is purchased from a registrar, the default values of the nameservers will point to the registrar's name servers. So after the initialregistration, bigbinary.com had the GoDaddy name servers, but has chosen notto use them. With the help of NS records, BigBinary has told GoDaddy thatBigBinary doesn't want to use GoDaddy's name servers and wants GoDaddy to usethe Cloudflare name servers. This will mean that all the future DNS records forbigbinary.com can be maintained in Cloudflare.

This goes on to show that one is not restricted to the name servers of theregistrars.

Name servers, just like CNAME records, never point to an IP address. They alwayspoint to another host name.

Different parts of a URL

Protocol

The protocol is the initial part that specifies the set of methods used fortransferring data between a web browser and a server. The server essentiallytells the browser how to communicate. The most common protocols are http(Hypertext Transfer Protocol) and its secure version https (Hypertext TransferProtocol Secure).

Top-Level Domain (TLD)

A Top-Level Domain (TLD) is the very last part of a domain name, located afterthe final dot.

For example, in google.com the .com is the Top-Level Domain (TLD). Common TLDsare .com, .org, .net etc.

Subdomain

A subdomain is a label that comes before the main part of a domain name.

For example, in the address blog.bigbinary.com the blog is the subdomain.

We can have a.b.c.d.e.f.blog.bigbinary.com. Technically upto 127 levels ofsubdomain is allowed.

Root domain

A root-level domain is the base name of a website. For example, inwww.google.com the .com part is the TLD and google.com part is the rootlevel domain.

Strange case of www subdomain

Let's say that we registered a brand new domain called spinjudo.com withCloudflare. We have our marketing site up and running on Vercel, and we want tohost our blog on WordPress. So we will add these two DNS records.

For our marketing site, we will add a CNAME record something like this:

spinjudo.com IN CNAME 76.76.21.21

And then for the blog, we will add another CNAME record something like this:

blog   IN   CNAME   domains.wordpress.com.

So now both https://spinjudo.com and https://blog.spinjudo.com are working.

But what about https://www.spinjudo.com.

Developers often forget that users can type https://spinjudo.com or they canalso type https://www.spinjudo.com.

Time to time in your life, you might have come across cases where a site isworking only with www or only without www. That's because the developerseither took care of www and forgot to take care of the root domain or thedeveloper took care only of the root domain and not of www.

Technically, www is a subdomain and that needs to be taken care of too. Inthis case for spinjudo.com we need to add one more CNAME record for www andthat might look like this:

www IN CNAME 76.76.21.21

Rules regarding CNAME records

The first rule is that CNAME records can only be added to subdomains. Itmeans CNAMES can't be added to root domains. That's the rule.

However, some DNS providers allow a technique to bypass this rule. They providea different type of record called "ANAME" record which allows CNAME for rootdomains.

In general, we advise against using ANAME records.

The second rule is that if a subdomain has a CNAME record, then that subdomaincan't have any more records.

Why do we need to wait for some time after updating the domain

If you change anything related to DNS you are asked to wait 24 to 48 hours forthe change to propoage. What does that mean?

If you type www.neeto.com into your browser, it uses its DNS cache. The browsermight not even ask anyone what the ip address of www.neeto.com is. This isbecause maybe 20 seconds ago you visited www.neeto.com and now you refreshed thepage.

If the browser really needs an updated DNS value, then the browser will sendthat request to your laptop. Your laptop has a DNS cache. Your laptop might notask anyone for the updated DNS value for a while.

Now, if the laptop needs to know the DNS valu,e then the laptop will ask for anansewr to the root servers.

There are only 13 root servers in the world. These root servers are named from"A" to "M". However just because there are 13 root servers it doesn't mean thatthere are only 13 servers. Each of those 13 root servers are running behind lotsof physical servers.

The complete list of physical servers can be found at https://root-servers.org.

Root servers do not know the answer to every question. However, they do know theaddress of the servers that can answer your questions. For example, if your TLDis .com, then the root server will send you to a different server. If your TLDis .net then you will asked to go to a different machine. Similarly, there areservers to serve TLDs like .co.uk, .in etc.

Note that at each layer, there is some amount of DNS caching. To ensure that theupdated value is completely updated throughout the Internet of the whole worldwe need to give some time for the old cached value to be removed.

The way these servers work also shows the decentralized nature of DNS. There isno one single place where all the DNS values of the world is stored. DNS valuesare stored at different places, but there is a path to get to those values allthe way from the root servers.

neeto-custom-domains-frontend package

Adding a custom domain could be intimidating. People are asked to change DNSrecords, which they might not have touched for years. A small misconfigurationhas the potential to bring the whole website down. To ensure that the userexperience of adding a "custom domain" is great, we atNeeto decided to go the extra mile.

We bought one domain each at the following name registrars:

• Cloudflare
• Namecheap
• Hostinger
• Digital Ocean
• Wix
• Porkbun
• Squarespace
• AWS route 53
• Network solutions
• Godaddy
• Strato
• Microsoft 365

We checked out how they ask their users to make DNS entries and we captured thescreenshots. Now, depending on what "name server" the user is dealing with wedisplay a help message and help screens customized for that user.

For example, if a user is using Cloudflare as their name server then the helpscreen might look like this.

Notice that the above image has a column called "Proxy" which is cloudflarespecific.

If the domain registrar is GoDaddy then the help screen might look like this.

At Neeto we build a number of products and almost all theproducts need the feature of "custom domain". To share the code related todomain handling in a consistent manner, we have built a utility tool namedneeto-custom-domains-engine.

When a user adds a domain to the Neeto product, this tool finds the name serverof that domain. Then we check if we have the custom help screen and helpinstructions for that name server or not. If we don't, then we provide generichelp instructions.

Validating DNS records

Now that the user has added the custom domain, Neeto needs to verify that thoseDNS records are indeed added.

If a user tries to add the domain www.spinjudo.com, then Neeto will ask theuser to add following CNAME for www.

Now, let's assume that the user has added that record. Now we need to verifythis record. Let's execute the following command from the terminal.

nslookup -type=CNAME www.spinjudo.com

We should see a response similar to this:

Non-authoritative answer:www.spinjudo.comcanonical name = dns.neetodeployapp.com.

Here We can see that the DNS record has propagated properly for this domain andit's receiving the expected value. Now, let's see how this would look for arecord without the proper records added.

nslookup -type=CNAME www.spinjudo2.com

This is the response we will get for spinjudo2.

Non-authoritative answer:*** Can't find www.spinjudo2.com: No answer

This could mean two things.

1. Either the user has not added the records properly.
2. The records have been added, but they have not propagated properly. Sometimesit can take upto 48 hours for the added records to be seen.

In this case we saw the example of validating CNAME records. However sameprocess is followed when it comes to validating A records or AAAA records.

Traefik to route the custom domain

Typically, when it comes to deploying an application one can deploy theapplication using services like Heroku, Render, Railways or directly on cloudservices like EC2, GCP, Azure etc. At Neeto, we decided to build NeetoDeploy.It's a service similar to Heroku for our internal use.

NeetoDeploy uses Traefik, which is a leadingmodern open source reverse proxy. In simple terms, Traefik is a load balancer.All requests come to the load balancer and then the requests are routed to theright place.

Earlier, we talked about setting up a custom domain. I have setup custom domainfor https://calendar.spinjudo.com. As part of setting up a custom domain, Iadded a CNAME for dns.neetodeployapp.com.

After adding the CNAM,E Neeto validated the record. Once Neeto determines thatthe user has correctly added the CNAME then Neeto adds these custom domains toTraefik. In reality, four records are added as shown in the picture.

In total, we see four records. The first records are "websecure". It means theyhave https turned on. The last two are just web. It means they supporthttp.

Once a request for a domain for calendar.spinjudo.com comes, then thoserequests will be sent to NeetoDeploy's Traefik. Traefik will map thatcalendar.spinjudo.com to an instance of the NeetoCal application and therequest will be forwarded to that server.

Please note that Traefik doesn't fulfill the request. Traefik acts as a loadbalancer and sends the request to the right server.

Finding the ip address

When a user types https://calendar.spinjudo.com, then the browser needs to knowthe ip address to which to hit. In other words, we ultimately need to know the Arecord.

We can use the tool dig to find the final A record value when a user visitshttps://calendar.spinjudo.com. First, let's see the tool in action.

dig +noall +answer calendar.spinjudo.com A

The response we get is following:

calendar.spinjudo.com. 60 INCNAMEdns.neetodeployapp.com.dns.neetodeployapp.com.60INCNAMEa7b4fc193275e43ea9ba2b7753b080dc-6ad6aaca90bd0ea4.elb.us-east-1.amazonaws.com.a7b4fc193275e43ea9ba2b7753b080dc-6ad6aaca90bd0ea4.elb.us-east-1.amazonaws.com. 60 IN A 34.233.85.113

The first line is a CNAME to dns.neetodeployapp.com.

The second line is a CNAME to an EC2 instance on AWS.

The third line is an A record, which gives us the ip address of the machine.

In the above command, we used +noall and +answer. The +noall is reallyuseful. It means "just hide all the output". By default, dig output spits alot of information. By using +noall we tell dig to go on quite mode.

And then we tell dig to show data for the "answer" section by using +answer.

Similarly, using dig +noall +authority will show only the data about the"authority" section.

Issuing SSL certificate

We know that a user can visit http://spinjudo.com or the user can also visithttps://spinjudo.com. To serve HTTPS, we need SSL certificates.

We can use the user to purchase SSL certificate, configure it and apply it butthat would be too cumbersome for our users.

To generate SSL certificate we will useLet's Encrypt which issues free trusted SSLcertificates.

SSL certificates are issued using the Automatic Certificate ManagementEnvironment (ACME) protocol by Let's Encrypt. This is a simple protocol by whichthe service can verify the authenticity and ownership of the requested website.

The process of domain validation, issuinga certificate and then renewingcertificate could be a bit daunting. To make life easier, Let's Encrypt hasprovided certbot. This tool makes it easier to getSSL certificate from Let's Encrypt.

It's worth noting that the certificate issued by Let's Encrypt is valid only for3 months. Before the certificate expires, we need to ask Let's Encrypt to issuea new certificate and that certificate will be valid for another 90 days.

NeetoCal using Cloudflare

NeetoCal and other Neeto products use Cloudflare astheir name servers.

If we look at the NeetoCal DNS record, then this is what we see.

Adding a * in place of the name of the record is known as a wildcardcharacter. This serves as a catch-all for any undefined subdomain, meaning allthe subdomains that do not have an explicit DNS record associated with it willbe matched by this record. As we can see in the picture above, the * value isproxied. What it means is that any attempt to connect to the NeetoCal IP addressfirst hits the Cloudflare server. Cloudflare server runs a bunch of checks onthe incoming requests, and then passes that request to the intended IP address.

For example, if I'm getting a lot of spammy requests from Spain, then I canconfigure Cloudflare to prevent requests coming from Spain. Or I can rate limitrequests for /login url and things like that.

If we do not use proxy, then that means the user is directly connecting to theNeeto server and Cloudflare is not in the picture. In this case, if aDDoSattack happens, then Cloudflare will not be able to protect us.

Heres a snippet illustrating the issue faced by our team:

// For requests, we had to manually convert camelCase values to snake_case.const createUser = ({ userName, fullName, dateOfBirth }) =>  axios.post("/api/v1/users", {    user_name: userName,    full_name: fullName,    date_of_birth: dateOfBirth,  });// For responses, we had to manually convert snake_case values to camelCaseconst {  user_name: userName,  full_name: fullName,  date_of_birth: dateOfBirth,} = await axios.get("/api/v1/users/user-id-1");

This manual conversion process consumed valuable development time and introducedthe risk of errors or inconsistencies in data handling.

To streamline our workflow and enhance interoperability between the frontend andbackend, we decided to automate case conversion.

Implementing automatic case conversion

Implementing automatic case conversion across Neeto products required athoughtful approach to minimize disruptions and ensure a smooth transition.Here's how we achieved this goal while minimizing potential disruptions:

1. Axios Interceptors for Recursive Case Conversion

We created a pair of Axios interceptors to handle case conversion for requestsand responses. The interceptors were designed to recursively convert the cases,managing the translation between snake case and camel case as data traveledbetween the frontend and backend. This smooth transition simplified theworkflow, cutting out the requirement for manual case conversion in mostsituations.

2. Custom Parameters to Control Case Conversion

To do a smooth rollout without breaking any products, and due to certain specialAPIs requiring specific case conventions due to legacy reasons or externaldependencies, we introduced custom parameters transformResponseCase andtransformRequestCase within Axios. These parameters allowed developers toopt-out of the automatic case conversion for specific API endpoints. Byconfiguring these parameters appropriately, we prevented unintentional caseconversions where needed, maintaining compatibility with APIs that requireddifferent conventions.

This is how we crafted our axios interceptors:

import {  keysToCamelCase,  serializeKeysToSnakeCase,} from "@bigbinary/neeto-cist";// To transform response data to camel caseconst transformResponseKeysToCamelCase = response => {  const { transformResponseCase = true } = response.config;  if (response.data && transformResponseCase) {    response.data = keysToCamelCase(response.data);  }  return response;};// To transform error response data to camel caseconst transformErrorKeysToCamelCase = error => {  const { transformResponseCase = true } = error.config ?? {};  if (error.response?.data && transformResponseCase) {    error.response.data = keysToCamelCase(error.response.data);  }  return error;};// To transform the request payload to snake_caseconst transformDataToSnakeCase = request => {  const { transformRequestCase = true } = request;  if (!transformRequestCase) return request;  request.data = serializeKeysToSnakeCase(request.data);  request.params = serializeKeysToSnakeCase(request.params);  return request;};// Adding interceptorsaxios.interceptors.request.use(transformDataToSnakeCase);axios.interceptors.response.use(  transformResponseKeysToCamelCase,  transformErrorKeysToCamelCase);

Note that keysToCamelCase, serializeKeysToSnakeCase are methods from ouropen source pure utils library@bigbinary/neeto-cist.

While rolling out the change to all products, we wrote a JSCodeShift script toautomatically add these flags to every Axios API requests in all Neeto productsto ensure that nothing was broken due to it. Then the team had manually wentthrough the code base and removed those flags while making the necessary changesto the code.

After the change was introduced the API code was much cleaner without theboilerplate for case conversion.

// Requestconst createUser = ({ userName, fullName, dateOfBirth }) =>  axios.post("/api/v1/users", { userName, fullName dateOfBirth })// Responseconst { userName, fullName, dateOfBirth } = await axios.get("/api/v1/users/user-id-1");

Pain points

In our work towards automating case conversion within neeto, we encounteredseveral pain points.

1. Manual work is involved

During the rollout phase of our automated case conversion solution, there was anunavoidable requirement for manual intervention. As we transitioned the existingcode bases to incorporate the new mechanisms for automatic case conversionwithin Axios, each Axios call needed an adjustment to remove the manual caseconversion codes written before.

This stage demanded some manual work from our development teams. They updatedand modified existing Axios requests across multiple projects to ensure theyaligned with the new automated case conversion mechanism. While this manualeffort temporarily increased the workload, it was a necessary step to implement theautomated solution effectively across Neeto.

This phase highlighted the importance of a structured rollout plan andmeticulous attention to detail. Despite the initial manual workload, once thechanges were applied uniformly across the codebase, the benefits of automatedcase conversion quickly became evident, significantly reducing ongoing manualefforts and improving the overall efficiency of our development process.

2. Serialization Issues

As our initial implementation of automated case conversion, we usedkeysToSnakeCase method, which recursively transforms all the keys to snakecase for a given object. It internally used transformObjectDeep function torecursively traverse through each key-value pair inside an object fortransformation.

import { camelToSnakeCase } from "@bigbinary/neeto-cist";const transformObjectDeep = (object, keyValueTransformer) => {  if (Array.isArray(object)) {    return object.map(obj =>      transformObjectDeep(obj, keyValueTransformer, objectPreProcessor)    );  } else if (object === null || typeof object !== "object") {    return object;  }  return Object.fromEntries(    Object.entries(object).map(([key, value]) =>      keyValueTransformer(        key,        transformObjectDeep(value, keyValueTransformer, objectPreProcessor)      )    )  );};export const keysToSnakeCase = object =>  transformObjectDeep(object, (key, value) => [camelToSnakeCase(key), value]);

However, this recursive transformation approach led to a serialization issue,especially with objects that required special treatment, such as dayjs objectsrepresenting dates. The method treated these objects like any other JavaScriptobject, causing unexpected transformations and resulting in invalid payload datain some cases.

To mitigate these serialization issues and prevent interference with specificobject types, we enhanced the transformObjectDeep method to accommodate apreprocessor function for objects before the transformation:

const transformObjectDeep = (  object,  keyValueTransformer,  objectPreProcessor = undefined) => {  if (objectPreProcessor && typeof objectPreProcessor === "function") {    object = objectPreProcessor(object);  }  // Existing transformation logic};

This modification allowed us to serialize objects before initiating thetransformation process. To facilitate this, we introduced a new method,serializeKeysToSnakeCase, incorporating the object preprocessor. For specificobject types requiring special serialization, such as dayjs objects, weleveraged the built-in toJSON method, allowing the object to transform itselfto its desired format, such as a date string:

import { transformObjectDeep, camelToSnakeCase } from "@bigbinary/neeto-cist";export const serializeKeysToSnakeCase = object =>  transformObjectDeep(    object,    (key, value) => [camelToSnakeCase(key), value],    object => (typeof object?.toJSON === "function" ? object.toJSON() : object)  );

This resolved the serialization issue for the request payloads. Since theresponse is always in JSON format, all values are objects, arrays, orprimitives. It won't contain such 'magical' objects. So we need this logic onlyfor request interceptors.

Conclusion

In simplifying our web development workflow at Neeto, automating case conversionproved crucial. Despite challenges during implementation, refining our methodsstrengthened our system. By streamlining data translation and overcoming hurdleslike serialization issues, we've improved efficiency and compatibility acrossour ecosystem.

If you're starting a new project, adopting automated case conversion mechanismssimilar to what we've built in Axios can offer significant advantages.Implementing these standards from the beginning promotes consistency andsimplifies how data moves between your frontend and backend systems. Introducingthese practices early in your project's lifecycle help sidestep thedifficulties of adjusting existing code and establishing a unified conventionthroughout your project's structure.

For existing projects, adopting automated case conversion might initially comewith a cost. Introducing these changes requires careful planning and executionto minimize disruptions. The rollout process might necessitate manual updatesacross various parts of the codebase, leading to increased workload andpotential short-term setbacks.