Inform HN: MailChimp blacklists your IP should you open the browser’s dev instruments

Phil Tadros · March 20, 2023

2023-03-20 12:56:59

I had to use their silly drag-and-drop email builder because I’m handing the email off to be edited by a non-dev. I dropped in a “Code” module so I could add some custom CSS but because a style tag generates no space, that module is no longer accessible via the UI as there’s nothing to click on. So I thought oh brother I’ll just inject a couple br tags via the Inspector and then poof, I’m in the doghouse.

Rather than it being dev tools itself, I think it’s more likely that your injected <br> tags got POSTed to an API endpoint (or similar) in unescaped format, and were categorised by a WAF as attempted XSS. It’s common for WAFs to block you for this kind of thing, unfortunately.

Still ridiculous, but not quite the same thing as being banned for opening dev tools (of course, I am also speculating here, I guess we’d need to hear from mailchimp to be sure).

Actually just opening the dev tools triggers it. The blacklist seems to expire on its own so I went ahead and opened the dev tools and did nothing more, reloaded, blocked.

This is interesting as something I’ve never thought about. What signaling to the server does the browser do when devtools are opened, and I guess I have to ask why is it signaling to the server anything at all if it is?

Edit: i see that people have replied with answers to this further down the page

This is worrying since I have accidently opened dev tools hundreds of times by clicking both mouse buttons when my cursor is near the bottom of the screen.

I have a tic disorder (not Tourette’s, because my tics are all nonverbal). One of my tics is that I mash both mouse buttons over empty space pretty frequently. I even go out of my way to keep my cursor positioned over empty space so I can mash the mouse buttons when I need to, and it’s not uncommon for me to move the cursor while mashing the buttons. If the cursor is towards the bottom of the screen, that’s pretty much guaranteed to open dev tools, since all it takes is a small motion of the cursor with the right-click menu open to hit the ‘Inspect’ option.

Likewise I hit F12 all the damn time because I’m aiming for the Home key, which is undersized on my keyboard, and they’re right next to each other.

Great, now I need to wait 8 seconds while my browser re-renders some 40-meg page which could’ve been plain text.

On the other hand, if I ever think about using MailChimp to send spam, I hope someone would just come cut my hands off, then I won’t need to care about hitting the wrong keys.

I regularly instruct users to open dev tools to clear their site-specific cookies because there doesn’t seem to be a way of doing this without clearing _all_ cookies anymore other than in Dev Tools > Application

This is why I despise gestural focused computing. Of all the features in any software, I think my least favorite is pull-to-refresh.

I suspect that would be easy to solve with smarter context menus that could ignore clicks likely to be accidental, since “Accidentally clicking the thing that just popped up before you even see it” is a common ish mistake worth implementing workarounds for.

It’s incredibly easy to do on a MB pro with a touch bar if you keep the function keys visible and tap the minus key with an open and relaxed hand. I preface my notes with — and == so I do it fairly often.

When you open devtools, by default it will try to load source code maps for your JS and CSS.

Very simple for a system to detect the request for the map file.

If that’s their vector turn off the autoloader and try from a clean IP.

Unfortunately console.log(foo) calls foo.toString() if and only if the console is open, and there is no way to disable this in Chrome or Firefox.

Edit: You can redefine console.log to be a noop, but that’s also detectable.

Interesting to note. In Firefox “show original sources” seems to be enabled by default but in Chrome at least the settings checkbox is labeled “Allow DevTools to load resources, such as source maps, from remote file paths. Disabled by default for security reasons” and unchecked for me. Haven’t checked Safari to see what its behavior is.

Hmm, I’m on Chrome 111 on Linux and there are two boxes for loading maps – one for JS and one for CSS.

Could yours be a Windows Group Policy from $WORK?

I have 3 boxes: “Enable JavaScript source maps”, “Enable CSS source maps”, and “Allow DevTools to load resources, such as source maps, from remote file paths. Disabled by default for security reasons”. The first 2 are checked but without the 3rd trying to load source maps doesn’t seem to do anything unless I have them locally. It’s very possible I’m just testing it wrong, I don’t use source maps often. It’s also very possible Firefox does something similar and I’m just overlooking the option/behavior there.

This is interesting as for a hackathon i was thinking of ways to identify this behavior too.

The source map requests was a more successful option. Also played around with “snap” resize but it was too agressive.

As for whatever the reason MailChimp would block your up is pretty ridiculous.

I can only imagine the amount of security pressure they feel since they are basically a backdoor into easily stealing one or more company identities once you pass the 2FA, with full address books of customers that will trust emails deployed through MC campaigns and blindly click on links in the emails sent out, so I am guessing they err on the side of caution and have tons of false positives instead of letting anything pass through or disrupt.

reading further down that page…

“We retain a law firm in the UK to consult on EU privacy issues.”

wouldn’t it be better to retain a law firm that’s actually in the EU? hiring a UK law firm for EU matters is no different that hiring a US law firm, or AUS, or whatever non-EU country

While that is true in the sense that the UK is no longer in the EU, I don’t believe UK law has yet diverged from EU law on any relevant privacy issues, so UK firms would still have significant experience in this area. A firm within the EU would be a better choice, I agree.

Reminds me of my experiences with UnitedHealthcare’s website. If I try to log in with Firefox + uBO I get mysterious permissions errors and “something went wrong” messages for the next few hours, even after switching browsers. Use Chromium from the beginning though and it’s smooth sailing. And of course their “tech” support is beyond useless about this.

Lots of websites make me disable content blockers on Safari too, or even not let me use Safari (maybe because of Apple’s Private Relay?).

The part I do not understand is even websites that verify you via 2FA do this, so I assume their goal is to track you no matter what.

That’s an anticompetitive move. If you need to switch senders for some reason, the inspector is the only clean way to get an email’s HTML into another ESP.

Is that true?

An email client like Thunderbird or Mail will save a copy of the email on your local hard drive, which will include the HTML. This isn’t something I do regularly, but would be first first response if I needed to see the HTML of an email. Maybe Mailchimp has protections against this route too?

Yes, it’s true. You don’t want all the chrome from the actual send around the body of your email because the other ESP will be providing that. You might also want to prevent certain fields and links from converting into the send versions. But in a pinch, sure, you could slice the body out of a copy in Thunderbird.

Yes it does. Say you’re an agency sending email on behalf of several different organizations. If you export one to send through CampaignMonitor (usually list or domain approval related), the employee who pulled the HTML gets their hand slapped by the IP ban. It’s less likely to happen next time with a different campaign or different client. I haven’t actually experienced the IP ban but I’ve sent for the same organization through multiple ESPs without quitting one for good.

Even if you are a single organization user and leaving for good, you might do so gradually or perform test sends first. Speaking from experience again.

Wow, so this is why I’ve been having trouble getting Mailchimp to load lately. As a developer I often have devtools open for whatever I’m working on. If need to help out marketing with an automation or something, using the same tab, I get banned for the day.

Would be funny if it were a corporate reaction to a security researcher contacting them about some silly web API design (e.g., endpoint taking an arbitrary account ID without authorization check).

In the writeup, the researcher illustrates by copying the service URL from the browser’s dev tools. And so the obvious corporate corrective action is…

Mailchimp previously cranked up pricing on their Mandrill product dramatically, with minimal warning, no opt in, and unsympathetic tone from C-suite.

Mailchimp is pretty hostile to developers. I don’t recommend using them after that experience.

I’m convinced at this point they are trying to slowly tank Mandrill to get people to stop using it.

Tons of downtime, worsening delivery problems, no active development — or support even — for years, worse pricing, …

Or have it open before you visit the page. Assuming they are detecting a resize.

But if they really detect a resize, anyone who actually does resize their page will be blocked as well.

Doing that seems a bit insane to me.

Almost everyone resizes windows by manually dragging the corner. This will generate a series of resize events rather than a single jump. This is a real technique, but it tends to be used by shady piracy sites that want to stop piracy of their pirated contents.

There’s another cute approach based on the fact console.log(foo) calls foo.toString if and only if the devtools is open.

I could open the history or bookmarks panel in Firefox and get an instant resize. Or click <Super>+<Left> to snap the browser to the side of my screen.

It seems that this would create far too many false positives.

it’s been some years, but I think it’s still possible to detect the viewport resize without a corresponding window resize, so the ‘docked’ DevTools can be inferred distinct from a resizing of the window.

Isn’t this spying on the user without user’s consent? Me opening dev tools to view the source is a private activity. Is it okay to collect this kind of info and use it to block access? Are there no regulations against this?

> Isn’t this spying on the user without user’s consent?

Yes.

> Is it okay to collect this kind of info and use it to block access?

No. It’s not okay.

> Are there no regulations against this?

Probably not.
In this case the remedy is to just use another service, or DIY.
And I also think that’s a pretty reasonable remedy, which will send a message to others considering such actions.

FWIW I would like to see regulations around intrusive spying on client machines via the browser or any other path.
Ideally we’d get new, specific legislation around it.
Something might also be done at the executive level at the FCC.
Legislation is unlikely because of America’s current flirtation with 3rd world style politics.

In terms of advocacy, I would assume that the EFF is of a similar view.
Other human rights groups would be supportive of such measures, since in addition to protecting consumers, they protect journalists and their sources as well.
The people against will be state security services and all businesses powered by a targeted ad engine.

DIY’ing what MailChimp provides is… a lot. From everything people have said running your own mail servers alone is close to a fulltime job, probably multiple fulltime jobs. Specifically, the aspect of dealing with spammers and unsubscribes and not getting all of your mail rejected by major email providers.

I don’t know how they’re doing it, but I assume they’re using data given to them by your browser, e.g. measuring the difference in height/width between the window and the viewport. Either that, or (as someone upthread suggested) they’re reading a request your browser sends for mapping files, which would again be information you provided. If anything, I guess your browser is the one spying on you, by providing this information. But, realistically, I don’t think it counts as spying either way. Hostile behavior on Mailchimp’s part, yes. Dumb idea, yes.

If it were a “private activity”, they wouldn’t know you did it. If your computer sends requests to their server about it, then it’s not really fair of you to expect them not to be aware of it.

Does the computer really send requests to their server when I open dev tools? I checked and I couldn’t find any request that was sent from Firefox or Chrome when I opened dev tools.

Aren’t they using a JavaScript based detection mechanism like listening for browser events on the client side or a latching on to debugger to pull this information? Sounds to me like they are going out of their way to pull private information from my system that I or my system or my browser had no intention of sharing with them.

I used ChatGPT to get a refund from Mailchimp when their CKText editor failed to load on International Women’s Day, preventing us from sending an email promotion that day.

I opened the web inspector to show the library erroring when Mailchimp tried out a to load it, and also provided a screencast.

I wasn’t blocked, but I did receive a refund.

So this must be a relatively new thing!

If I were the OP, I would complain and request a refund.

Are you seeing any network requests or data transmitted over open websockets to confirm this is actually the case?

I get it that people like to complain about stuff on HN (I recently had a thread too) but there needs to be evidence for people to go off of.

Open the inspector, reload the page and you’ll see:

“Request Blocked

We blocked your request because the IP address you’re using looks suspicious. This issue will usually resolve itself after a short period of time, and you can try your request again. You can also try using a different IP address to see if that resolves the issue.

If you need additional help, you can try one of these support options.

Reference Number: #####”

Thank you for the description. I assume they’re using a bot management script which is set to block requests if devtools is open. For such websites, opening devtools in a separate window should work.

Some websites will try to throw you into a loop of debugger statements if they detect devtools being opened, which is harder to work around, but it doesn’t seem to be the case here.

Disabling javascript breakpoints usually does the trick. Devtools detection is often done by having a `debugger;` statement somewhere and timing of it triggered

A lot of them dynamically generate new anonymous functions to get around this. Last I looked (~1 year ago I think) neither Chrome nor Firefox supported disabling the keyword completely. Do you know if that’s changed?

On Firefox its in the Debugger Tab of the DevTools. On the top right you can deactivate all breakpoints (which includes the debugger statement).

There is a similar button on chrome, but I am not sure if that also applies to the debugger statement.

Couple thoughts:

1. Correctly designed dev tools shouldn’t be detectable from the app itself, especially not if the tools are passively used for observing. This can be abused by malicious actors who can make it harder to detect and warn others. It can also cause heisenbugs.

2. One if those malicious actors is apparently Mailchimp. I don’t use it so I’m not affected. But from a meta-perspective it’s concerning when direct user-hostile actions are normalized by what most people consider “legit companies”. The same could be said about fingerprinting and many other tricks.

3. Meta-meta point: if you’re running a business that does this, the open web is not for you. You don’t belong, and you should try building your own proprietary stack instead. I don’t mind wolves, but please stop dressing in sheep clothing. There’s a paradox of tolerance at play here.

I have seen websites that leave a few “debugger” keywords in the code and then use timing code to detect if you have the dev tools open. that is, if it takes too long to get to a check point, as in a person had to click resume, you know the debugger is open. It is very crude method but I guess was the best they came up with.

On firefox the easy way to get around this is by disabling breakpoints, the harder way is a userscript.

Like those right-click pop ups you used to get to prevent you copying images. We’ve come full circle. Only this time it’s a large profitable company rather than some random Geocities page.

Well that is poopy. Is there a way I can “stealth” open dev tools on all sites? I like to see network requests in a lot of places, but don’t like to think the server will change their responses based on my local actions.

There are a couple of ways they can try to detect devtools being opened. As the sibling comment implies, the most popular way is to detect a sudden viewport resize, and you can avoid that by ensuring your devtools are set to open in a new window before opening them.

The only other ways I’m aware of are:

– Detecting the keyboard shortcut, ⌘⌥i or equivalent, which you can avoid by using the browser menu, and

– More riskily, evaluating a `debugger` statement and detecting whether evaluation paused. I’m not sure you could do anything about this one, but it would certainly be obvious to you whether it was happening.

It’s pretty common as an account hijacking vector, right?

Hackers tell non-techies to paste things into the console, which can then share cookies or access tokens with the attacker.

Obviously the browser is “owned” by the client, so a sufficiently motivated techie could bypass this any number of ways. But it prevents some number of non-techies from security issues.

You can print debug messages to the console to warn people. The website has no idea if they opened it or not.

But here, it seems they are detecting if the window is resized! That’s just crazy.

I don’t want to believe they are that stupid. Why would a platform heavily used by developers make such an idiot? This is so funny right now. The most absurd security measure I’ve ever seen

You create this pseudo function:

1. setTimeout to call an endpoint that will block your ip. It’s set to run in 2 seconds from now.
2. Insert: debugger;
3. Clear the timeout.

Now you run this function as an interval.
If the devtools is closed, debugger will be ignored and the call to this endpoint will never happen. But if it’s open, the debugger will stop and the timeout will fire.
Not sure if you need to patch
SetTimeout to continue running while you stop but I hope you get the general idea

As others have mentioned, you can simply put in a sourcemap entry in your JS, and when the browser requests that URL (which happens when Devtools opens to prepare for showing original sources), ban the IP.

That would create far too many false positives. The common way is to detect if window.innerWidth changed by a minimum threshold but window.outerWidth did not. The limitations of the method are built in sidebars (such as on Edge with the sidebar and search sidebar) could trigger it as well if your minimum width is not wide enough while on the other hand if your minimum width is too wide you won’t catch the dev tools opening. The method is also limited in that undocked dev tools will not register a triggering change.

I was thinking the same thing. If I was intent on using the dev tools for who knows what with MailChimp, it would merely be a roadblock. I might even be more compelled to achieve my goal just to defeat their bullshit.

Not that I would ever use MailChimp.

Source Link