The Referer Header

The Referer header is what powers Formcake's whitelisting feature but what is it and how does it work?

When you click a link on a webpage, the browser sends a request to the server for the page you're trying to access. Along with this request, the browser also sends a header called "Referer" (often misspelled as "referrer") that indicates the URL of the page that contained the link you clicked. In this blog post, we'll explore what the Referer header is, how it works, and why it's important.

What is the Referer header?

The Referer header is an optional HTTP request header that specifies the URL of the webpage that linked to the current page. It was first introduced in 1996 as part of the HTTP/1.1 specification, and has since become a standard feature of all modern web browsers.

The Referer header is spelled with one "r" and two "e's", which is a common source of confusion. The correct spelling comes from a misspelling of "referrer" in the original HTTP specification, which was never corrected.

How does the Referer header work?

When you click a link on a webpage, the browser sends an HTTP request to the server for the page that the link points to. The request includes several headers, including the Referer header, which contains the URL of the page that contained the link you clicked.

For example, let's say you're on a webpage at the URL https://example.com/page1 and you click a link to https://example.com/page2. When the browser sends the request for https://example.com/page2, it includes the Referer header with the value https://example.com/page1.

The server can then use this information to track how users are navigating through the site and where they're coming from. This can be useful for things like analytics and debugging, as well as for preventing certain types of attacks.

Why is the Referer header important?

The Referer header has several important uses in web development and security. Here are a few examples:

Analytics: By analyzing the Referer header, website owners can track how users are navigating through their site, which pages are most popular, and where traffic is coming from. This information can be used to optimize the user experience and improve the site's performance.

Debugging: When a user reports a problem with a webpage, developers can use the Referer header to see the exact path the user took to get to the problem page. This can make it easier to identify and fix the issue.

Security: The Referer header can also be used to prevent certain types of attacks, such as cross-site request forgery (CSRF). CSRF attacks occur when a malicious site tricks a user into submitting a form on another site without their knowledge or consent. By checking the Referer header, the server can ensure that the form submission came from a legitimate source.

Limitations and concerns

While the Referer header can be a useful tool, it's important to be aware of its limitations and potential privacy concerns. Here are a few things to keep in mind:

Not all browsers send the Referer header: Some browsers, such as Tor, have the Referer header disabled by default for privacy reasons. This means that website owners may not always be able to track how users are navigating through their site.

The Referer header can leak sensitive information: If a user clicks a link to a page on a different domain, the Referer header will include the full URL of the previous page, which could potentially leak sensitive information. For example, if a user clicks a link to a health website from a page about a specific medical condition, the Referer header could reveal that the user is interested in that condition.

The Referer header can be spoofed: It's relatively easy for an attacker to spoof the Referer header and make it look like a request came from a different page. This can be used to bypass certain security measures that rely on the Referer header.

Conclusion

The Referer header is a useful tool for web developers and security professionals, but it's important to use it responsibly and be aware of its limitations. By understanding how the Referer header works and what it can and can't do, website owners can make informed decisions about how to use it to improve their sites' performance and security.