Broken things in the Internet

Random Abstract Findings

May 24th 2021

Some ramblings about annoying things in the Internet today, mostly about security and privacy, sprinkled with some wishful thinking at the end. This is a bit of a rant, but there are some useful parts in it too.

1 Problems

Here is a list of problems with Internet services today. These are all loosely related to each other. Some are about privacy, others about authentication processes.

1.1 Online services require an e-mail account

Most online services require an e-mail address to be associated to the account. This email address is used for several purposes:

  1. Verification: the account is not available until you verify you own the e-mail address by checking a confirmation code
  2. Recovery: if you lose your password then you can get a recovery e-mail with a process to get back into your account
  3. Pseudo 2 Factor Authentication (2FA): we will send you a temporary code before you can login - even if your e-mail is also your password recovery mechanism
  4. Notifications: sends you information about the service

The verification step is just meant to verify you own the e-mail account before the account is activated and more e-mail can be sent to the address. i.e. Verification is not really a feature, but a security requirement before you can implement other features via e-mail.

Pseudo 2FA is interesting in that implementing 2FA with a password+email code is not enough, if holding control of the e-mail account also enables password recovery. Either Recovery needs an additional authentication factor, or 2FA should NOT rely on the regular password. Also introducing a requirement on having e-mail for authentication, means your availability is now dependent on the e-mail service.

E-mail notification handling is basically outsourcing of notifications to public infrastructure (i.e. e-mail servers). There are services out there that serve the bulk of their notifications via internal APIs (e.g. in mobile applications) and don’t actually use e-mail for this. So this is basically a matter of convenience, the user already has an e-mail account and gets notifications via this channel. There are plenty of other notification channels out there (Whatsapp, IRC, whatever notification protocol works these days).

I should also point out here that, e-mail is not particularly good at distinguishing between good e-mail and spam. Which is why most of our automated service e-mails end up in the spam box.

1.2 Email services require phone numbers

E-mail service are a specific case in that instead of requiring an e-mail address they require a phone number for account creation. Privacy is the main issue here - why is this service requiring us to add a phone number? The primary reason are similar to other services, the phone number is used when:

However it is now affordable to get temporary phone numbers to get confirmation codes. There are even some free services to get SMS messages[1] in temporary numbers over multiple countries.

Some oddballs in this regard:

Basically account verification/recovery offloads control of authentication to a phone provider If the phone company takes over, or you lose control over your phone number, you may be locked out of your e-mail

The potential benefits for the e-mail service are:

1.3 Completely Automated Public Turing test to tell Computers and Humans Apart

The use of Captchas for abuse prevention is now widespread. Here is a list of scenarios where these pop up:

There are two main reasons to use them

Sadly Captcha execution is now a sure way to annoy humans - you end up spending 15min clicking fire hydrants and semaphores, just to login into an account. It is worth pointing out that Captcha options are not uniform across services, and some Captchas are easier on humans than others.

There are three questions to answer here:

If you are concerned with rate limiting, there are better ways to do it:

Concerning proof of humanity - I always find it amusing when my mobile operator website forces me to go through Captchas when logging into their website. It looks like they don’t realize they have other ways to reach me for verification.

In the cases when the Captcha comes after authentication - the order seems wrong. If the intent was to rate limit by verifying a human is present, then authenticate the user after proof of humanity is done. There are a couple of exceptions to this - but login and search are definitely not the cases where you need proof of humanity.

Another factor that does not help is forcing to refill the Captcha every time a form is submitted - which again supports that what we are doing is rate limiting not proof of humanity.

1.4 Universal 2nd Factor authentication

U2F is, so far, an undelivered promise. Primary because adoption has been so low in services, browsers, and other applications.

Currently there are two main uses for it:

In principle U2F has great potential, both as second authenticator factor or even as main authentication - it is definitely more secure than putting passwords in forms and resistant to regular phishing attacks. However, there is nothing preventing you from fully automating U2F responses, and generally U2F does not work as proof of humanity

In practice we only saw U2F in use via browsers and the javascript API. But it could be used in other protocols, e.g. OpenSSH already has U2F support. Frankly browsers and javascript should not be a requirement for any type of authentication. I have yet to find a proposal for TLS session authentication using U2F outside of academia [3].

Cloudflare recently published an article [4] where they promote the replacement of Captchas with U2F. To be clear here, in the same post they restrict the U2F tokens to a small list of trusted devices - precisely because U2F (the protocol) can not ensure you are human, the best you can do is trust certain tokens are trustworthy for this purpose.

There are also other limitations. U2F cannot be used in cases when the user is notregistered - so it is not a general purpose rate limiting solution, but it may be a user scoped one.

One interesting bit about U2F, is that the U2F authentication usually happens as the second factor in authentication. This is interesting if you are concerned about phishing attacks stealing your regular passwords. Shouldn’t we be doing U2F before asking the user for manual passwords?

2 Discussion

So what can we do about these problems? First, I think we should break down the roles fulfilled e-mail/phone and start introducing better alternatives, in such a way that:

2.1 Break down user privileges per scope

The default model for web sites is that the user holds one password, and this credential is all powerful. Losing this password to an attacker likely means losing the account for ever, because this password can subvert the password recovery mechanism. Forgetting this password, requires creating a new password through some mechanism the service provider registered during account creation - which in turn depends on another credential (e.g. e-mail, master password, or a security question).

First lets separate these two roles

If the first one is compromised, use the second one recover the account.

We could continue this break down even further:

There is nothing particular new here. There is an usability issue in that users are not used to managing multiple passwords per account within password managers.

2.2 Provide the user with an Inbox

Services can provide you with an inbox for service activity notifications. This inbox can be queried via an API. Preferably these APIs are standard enough to avoid reinventing the wheel (e.g. even RSS works for this).

A side note here is that IMAP is no different than any other API - the problem with regular e-mail is that the destination server is ,usually, not owned by the service provider or by the user. This means a third party is now involved in the process. As a funny thought experiment a service could provide an IMAP server that exposes service activity as e-mails.

This removes the dependency on external services for a notification inbox.

E-mail notifications sent to external services should be optional. They are a convenience, similar to other messaging protocols such as Whatsapp or SMS and are NOT special.

Do not use this inbox for one-time password recovery codes. For two reasons:

It can however be used to notify you about account lockdowns, or that recovery is in progress.

2.3 Stop outsourcing Rate Limiting

Out sourcing rate limiting is a bad idea.

What exactly is the maximum rate for a user to perform this action? This sounds like something specific to your service. The same is truth for security - how often do you want a user to attempt a login? This depends on the authentication method, use-cases, your risk profile and so on.

Implementing rate limiting in your service is not that hard. The main decisions you have to make are about the scope of limits i.e.:

Captchas are not a rate-limiting mechanism, stop using them as such. They are a mechanism to prove a human is on the other side - and even so the machines are getting better than humans at solving them. They should not be your single rate-limit on sensitive operations.

2.4 Proof of humanity should not punish humans

I don’t have data to support this - but I think we fill way more Captchas than we should. If a human makes a mistake in a form, his reward is an input error and another Captcha, even if he successfully filled another Captcha 2 minutes earlier.

It is a lot more reasonable to ask people to be patient and wait 2 minutes before retrying than to throw them more challenges. That is, limit the rate of the Captchas you throw at users, and simultaneously rate limit user actions in time - manage these two limits separately.

3 Things I would like to see happen

3.1 Public service e-mail

I would like to see the post office providing citizens with e-mail inboxes, even if those inboxes are restricted to a list of domains that can send them e-mail.

My reasoning here is that we are now quite dependent on e-mail for services we consider essential. My power, internet and trash services regularly send me e-mails with bills or updates. Where should I be receiving this e-mail.

I want to be clear here, since this is a public service:

This list of requirements will make it impractical for a lot of purposes, and that is perfectly fine.

3.2 Free services without dependencies

Reddit is likely the most popular service that still allows registration without e-mail addresses or phone numbers. We need more services that break this dependency, but we also cant take the same approach as reddit, where your single means of authentication is a password, and loosing your password means you loose the account.

For our purposes we want to reduce dependency on external services, and increase security. We want to avoid storing secrets and state on the service side as much as possible, to reduce the impact of spam and data leaks.

Signup consists of registering a public key with the service. In fact we register two keys one for account recovery and another for regular use (the service links them together). Let us assume 64 bytes of state per account, which may look small, but will pile up with a large number of users. So I would recomend a proof of patience (or work) to curb the registration of spammer accounts. A proof of work here MUST require the use of the key, to avoid users registering keys that they do not own.

NOTE: Another options here is to have the service sign user public keys, generating a certificate, but this requires the user to store this certificate instead of the service.

It is up to the user to securily hold the corresponding private keys. If these are lost then the account is also lost.


“Receive SMS.” Available:
“Proof-based authentication for HTTP messages.” Available:
“Humanity wastes about 500 years per day on CAPTCHAs. It’s time to end this madness.” Available: