Client IP Address Disclosure in various consumer mail servers
Thursday, 2 April 2020
Summary
When email users of several email services send mail using mechanisms other than that service's web interface (i.e.: their phone or laptop’s email program), services commonly include the user’s IP address in message headers. This information disclosure lets recipients of these messages perform some privacy-invasive actions, such as:
-
- Approximate geographical location of the sender
- Correlation of separate email addresses, but sent by the the same sender
- Broadband and / or cellphone provider
Users looking to send email in a manner that keeps this information private from message recipients should use either the web interface or an alternative mail provider.
Impacted:
- Gmail
- Google Suite
- Apple (@mac.com, @me.com)
- Office 365
- Outlook.com (consumer) in uncommon configurations
- Most ISP email providers
Not impacted:
- Yahoo, AOL
- Outlook.com (consumer) in most common configurations
- Protonmail
Edited 4/13/2020 - A previous version of this post singled out Gmail. While this is the most popular email provider impacted by this flaw in common configurations, they are not the only one.
Impact
While the RFC requires that email programs identify themselves to SMTP servers, consumer mail is a relatively new thing as far as internet time goes. It is a change in threat model compared to the work-centric mainframe systems that were at the core when the RFCs were written. Today, people use consumer email systems in privacy sensitive situations that they didn’t before - whistleblowing, at-risk communities, avoiding domestic abuse, and more.
People that are uncomfortable with a mail recipient knowing their approximate geographical location should avoid using email applications to send email via the impacted email services, or use a privacy-preserving consumer email service.
Discussion
Non-web email clients generally use the Simple Mail Transfer Protocol (SMTP) to connect and send mail via their mail service provider. Examples of these non-web email clients are the mail applications built into phones (Android, iOS) or desktop / laptop operating systems (Apple Mail, Windows Mail).
When these applications connect to their mail server's SMTP server to send an outgoing mail, part of the protocol requires a “HELO” or “EHLO” message. RFC 821 describes this exchange as:
At the time the transmission channel is opened there is an
exchange to ensure that the hosts are communicating with the hosts
they think they are.
The following two commands are used in transmission channel
opening and closing:
HELO <SP> <domain> <CRLF>
QUIT <CRLF>
In the HELO command the host sending the command identifies
itself; the command may be interpreted as saying "Hello, I am
<domain>".
-------------------------------------------------------------
Example of Connection Opening
R: 220 BBN-UNIX.ARPA Simple Mail Transfer Service Ready
S: HELO USC-ISIF.ARPA
R: 250 BBN-UNIX.ARPA
Example 5
-------------------------------------------------------------
Most mail clients include the user’s public IP address and host name as part of this exchange, with Apple Mail even including the computer name and IP address on the internal network used to send the mail. One could argue that mail clients should not send this information as part of the exchange, but some SMTP servers validate that the actual IP address of the sender matches the content of the SMTP exchange as a form of spoofing protection.
As mail servers transport the message from its original SMTP server to the destination email server, much of this conversation is retained as part of the message’s mail header content. Microsoft has a good resource on how to see and interpret these mail headers here: https://support.office.com/en-us/article/view-internet-message-headers-in-outlook-cd039382-dc6e-4264-ac74-c048563d212c, but an example looks like this:
As you can see in the “Received: from” headers above, some SMTP severs redact the sender’s client IP out of the rest of the exchange, replacing it instead with the address of the SMTP server. Two examples of email services that perform this client IP redaction are Outlook.com (as above) and AOL.com. However, as noted above, there are some scenarios in which Outlook.com does include this information.
The SMTP servers of some services do not perform this redaction, which provides the following data to recipients of the message:
Recommendation
Users looking to send email in a manner that keeps this information private from message recipients should use either the service's web interface or an alternative mail provider.
Disclosure Timeline
Google:
March 9, 2020 – Reported to Google security
March 10-19, 2020 – Assisted with steps to reproduce
March 20, 2020 – Resolved by Google as “Won’t Fix (Intended Behavior)”
March 20-31, 2020 – Worked to confirm that there were no future plans to fix
April 2, 2020 – Got confirmation that the team was aware of the issue and has no plans to fix
Microsoft:
March 9, 2020 - Reported to Microsoft security
March 19, 2020 - Got confirmation that redaction was intentional for Consumer Outlook. This blog post and responses to it uncovered scenarios where this redaction wasn't applying, and this is still under investigation.
Apple:
April 3, 2020 - Reported to Apple security
No. 1 — April 3rd, 2020 at 1:45 am
Lee- Just remove this blog post, this is embarrassing that you don’t understand how email clients work.
You realized this is the same thing as web servers being able to see the IP addresses of web browsers?
You had good intentions but are totally off base by a hundred miles, just delete this blog post and pretend you never wrote it. You’ve wasted everybody’s time.
No. 2 — April 3rd, 2020 at 2:10 am
Visiting a website is different than emailing a person. You’re probably comfortable assuming that http://gmail.com won’t abuse your IP address. But whistleblower scenarios? Abusive relationships? At-risk communities? Much different.
No. 3 — April 3rd, 2020 at 3:28 am
Or simply use a VPN while sending email.
No. 4 — April 3rd, 2020 at 8:06 am
Not only this article is wrong, it has straight up lies in it. This is attacking gmail for no sane reason, Outlook.com works the same, it also discloses user IP when using SMTP, it does not redact it. I just tested this. Saying it does not, is straight up lie! https://gist.githubusercontent.com/xvybihal/527cc4fc1018a18319b2856a53710448/raw/0a2d4b90be927d3d06698814891d4f34300d85b7/outlook
No. 5 — April 3rd, 2020 at 11:20 am
Lee, I think you should remove all references to Gmail and Google in this article, as this has NOTHING to do with them. If you had read the RFC, you would realise the IP data is included no matter what email client or provider you use, including sending from Gmail Web interface (it just shows the IP of the server running the Web interface) the most likely reason Google said they wouldn’t do anything about it is it will violate the RFC and make tracing the source of spam emails impossible!
Again just to make it clear to anyone reading this based on the click bait title, this has nothing to do with Gmail and Google.
No. 6 — April 3rd, 2020 at 11:31 am
U’r wrong lee,
Lets take for example a mobile cell tower , all clients that connected to that tower are behind nat, meaning that 1k+- uses the same ip , even the cell provider wont be able to know who is who with a port.
This comment is on top of what the anonymous tip said
No. 7 — April 3rd, 2020 at 5:51 pm
Seriously, if you read RFC you would know how email works and what you call leak affects every SMTP server in the world, by the very design of SMTP protocol.
I’d follow the advice in first comment and just delete this post.
No. 8 — April 3rd, 2020 at 6:31 pm
Are you serious?
No. 9 — April 3rd, 2020 at 11:35 pm
If your dependent on your IP remaining secret for anomity you’re not worrying enough. A dedicated person can always get it back through trackers or just a warrant to Google.
Additionally, how is GMails SMTP Servers to differentiate between a client connecting and a regular ol email relaying to it? Both talk over the same SMTP paradigm.
No. 10 — April 4th, 2020 at 12:15 am
You realise this is a fundamental operation of SMTP, right? Maybe read a book or two first. Usually better to keep your mouth shut and look stupid, rather than open it and remove all doubt.
No. 11 — April 4th, 2020 at 1:30 am
I agree with Lee. It is always better for users of an email service to understand what metadata they are providing to the recipient of the message. There is a practical difference between sharing your IP address with the service provider and sharing it with the recipient of your email message, which may or may not matter in certain situations. It is almost certain that most users of Gmail’s service are not aware of this. Thank you Lee for clearly stating the situation.
No. 12 — April 4th, 2020 at 6:46 am
This is not a vulnerability; this is how email works and has always worked. Using webmail won’t help you either – many webmail providers will intentionally add a “X-Originating-IP” to show the IP address of the sender. Email would be more dangerous if you couldn’t see the sender’s IP address, as it would make it more difficult to validate a sender’s identity.
I would recommend removing this post to prevent the spread of misinformation.
No. 13 — April 4th, 2020 at 8:38 am
Thank you for sharing! This actually popped up as a suggestion on my Google Now.
Don’t listen to the previous comment. It’s like e-mail and SMTP works but it’s not how Google implemented it in Gmail (where it’s hidden). I wasn’t aware of this and like you mention this can be used by the recipient to see from which city you sent the email from for example.
No. 14 — April 4th, 2020 at 9:36 am
e-mail has for the biggest part of internet history contained the sender’s IP address.
It’s only since webmail that this information has been withheld.
This is just normal functioning of the protocols and no fail whatsoever
No. 15 — April 4th, 2020 at 3:33 pm
I just reported to apache a bug that access.logs contain client ip and user agent 😂
No. 16 — April 5th, 2020 at 3:12 pm
This is well-known, affects most mail providers, and OF COURSE the client IP *SHOULD* be within the headers. It’s a bad configuration if you webmail service hides the originating IP.
Email has never been, is not, and doesn’t have to be secure or anonymous communication.
Yes, I really think email should die. But as long as it’s alive, we should stick to its protocol.
Read the relevant RFCs, and then please delete your entire post.
No. 17 — April 13th, 2020 at 8:48 pm
@Adnan – I think visiting a website is different than emailing a person. You’re probably comfortable assuming that gmail.com won’t abuse your IP address. But whistleblower scenarios? Abusive relationships? At-risk communities? Much different.
No. 18 — April 28th, 2020 at 3:40 pm
This is a problem of your PC mail client, not Google’s. Indeed, I think you can hide your IP in mail header by either deleting it (not sure gmail will accept this though) or using a proxy or dedicated server (and VPN to it) etc… And again that was already asked!!!! https://superuser.com/questions/878500/is-there-any-way-to-hide-my-ip-address-in-email-headers
No. 19 — April 29th, 2020 at 12:53 am
Sorry Lee the first commenter is right. I know you’re a smart guy and that the security thing is somewhat new for you, but this post really does embarrass you. I have enjoyed reading many other things you’ve written, but this makes me cringe. You should speak to a solid security person you trust before posting something like this that diminishes your credibility.
Feel free not to release this comment from moderation, I don’t think it needs more publicly posted comments calling you out, I’d be happy if it just helped you see your error.
No. 20 — April 29th, 2020 at 7:49 pm
“consumer mail is a relatively new thing as far as internet time goes”
No mate, by internet time email is an ancient institution, by turn praised and reviled.
No. 21 — May 6th, 2020 at 9:43 am
[…] die Client-IP des E-Mails-Senders und veröffentlicht einen Blogbeitrag, der das als problematische Schwachstelle bezeichnet. In den Kommentaren wäscht man ihm ordentlich den Kopf für die Entdeckung des […]