We are writing to inform you about an "exploit" that our friends at Nostra (specifically, Nostra CTO Joseph Higgins) were kind enough to research, document, and bring to our attention. (If you’re wondering who Nostra is: they’re not a security company. They’re a site speed company, offering a portion of what we provide to our customers.)
As all e-comm professionals are aware, ensuring that your store and all of its pages are quickly indexed by search engines like Google is a major challenge. If your pages are slow to load, depend on JavaScript rendering, or, more generally, require excess computational resources from the index bot (e.g., GoogleBot), then this can’t happen efficiently—and consumers looking for information on Google are liable to miss your offerings.
To address this problem, in 2017 Edgemesh built an SEO optimization feature into our suite that pre-renders your pages, allowing search engines to crawl even more pages in less time. This tool has been in production for seven years, across hundreds of billions of pages views, and we've never encountered a security vulnerability. And yet … the Nostra team left no stone unturned.
Nostra's engineering team has devoted significant time and resources to discovering that this optimization has a potential "exploit." Nostra informed us of this possibility yesterday, and suggested that, if we did not offer a remediation within 24 hours, they would “reserve the right to proceed with public disclosure.” We see no reason to wait. In the spirit of radical transparency, and with thanks to our friends at Nostra, we are sharing the full details of this alleged “exploit” here:
Cache Poisoning for Pre-render payloads:
- IF there is no traffic on your site for a period of at least 15 minutes (a circumstance that we have never observed, on any customer's site, in more than ten years—but one that is, of course, possible)
- AND IF no other user visits the page in that time (as pre-render data is generated by every visitor to the page, and the most recent render takes precedence)
- AND IF a malicious actor was the last person to insert “bad” pre-render data
- AND IF that malicious actor has found a way to evade rate limiting from Cloudflare on the endpoint (since they need to continuously insert new bad data),
- AND IF that malicious actor has bypassed bespoke security rules at Cloudflare (which may be difficult since the required speed and repetition is a job for a bot),
- AND IF that malicious actor can override any subsequent "real" pre-renders (i.e., it must keep inserting its malicious render to remain “on top” … across all pre-render types [there isn’t just one cache])
- AND IF Google crawls your site at exactly that time that the bad actor’s data is the most recent
- THEN that malicious actor could send bad page details to search engines, which will appear in searches conducted by users—until they are updated by Google (which, according to our customers’ data, happens ~14,618 times a day, consistently spread throughout the day)
Like other vanishingly uncommon events – such as being struck by lightning during a shark attack, hitting red 20 times in a row in roulette, or winning the lottery on consecutive weeks – this is an actual possibility.
If you are an Edgemesh customer, and you would like to eliminate this risk, you can simply go to our portal and toggle off “Enable Prerender” —as you have been able to do since we introduced this feature seven years ago. (Though, as mentioned above, the "exploit" that Nostra has discovered has never been “exploited” in that time.) Of course, this will result in a decrease in your SEO efficiency—and, like us, Nostra recognizes the importance of SEO efficiency.
You are welcome to consider the trade-offs, and act accordingly.
Again, we would like to thank our friends at Nostra for bringing this vanishingly small and obscure risk to our attention. In an abundance of caution, we immediately implemented a fix against the specific “exploit” that they brought to our attention (although the examples they sent to us all used our debugging tools, since it’s nearly impossible to catch a pre-render you have sent.)
As a small and dedicated team with a deep focus on some of the largest customers on Shopify, we do not have the leisure to explore our competitors' offerings in such incredible detail and we have been remiss in performing a similar deep-dive into Nostra’s software.
Instead, we've devoted our time to making our customers' sites faster and better performing; leveraging innovative marketing attribution models, operational failover controls, traffic steering, and, most recently, building new forecasting tools (see forecast.edgemesh.com). And we’ve done all of this while providing our customers with the insights and analytics they need to understand what’s working, and what needs to change.
Unfortunately, we do not expect that we will have the luxury of changing our priorities in the near future, and will not be able to reciprocate and devote comparable resources to a review of Nostra’s code.
