Why Using Dummy Email for Testing Just Doesn’t Work

Testing email workflows might not be the most fun part of a QA professional’s life. It usually requires a lot of manual work setting up test email accounts. It needs to be done rather quickly to enable focus on more important aspects of the user’s journey. And, in the end, something usually goes wrong.

A single email issue can ruin even the best first impression users have after signing up. Why would an amazing UI or robust core feature matter if users can’t even confirm their account and log in? Or, they get locked out of their newly created account and can’t reset the password due to a malfunctioning link.

To avoid such issues, QA analysts and developers create dozens of dummy emails and try to test their workflows before a product is shipped. It usually brings decent enough results so they repeat it for every new project. Is there a better way to go about it, though? Let’s see.

How do dummy emails work?

The concept of temporary email accounts is simple. You create them with one of the dedicated services or by settings things up on your own. You get a unique email address that can receive (and sometimes also send) emails and can be used for any purpose. Due to its temporary nature, this address stops working after a specific time. Frequently, the lifespan of such an account ranges from 10 minutes (hence the commonly used term ‘10minutemail’) to a few hours. Once the account is deactivated, the inbox gets erased and the address becomes available again.

People create temporary emails mostly to stay anonymous on the internet. Email addresses are frequently required to use an app, take part in a promotion, or ask a question on a public forum. If you were to leave your private email, you would likely get tons of spam later on or your address could be used for more malicious purposes.

The other idea behind using temporary emails is to test email workflows. QAs or developers set them up and then prompt their application to perform a specific action that will result in an email being sent to a disposable account. If an email is received, they check if it displays as expected and if links and media work fine. If everything checks out, they move on to another account, testing similar workflows. On complex platforms with multiple email channels, this can take quite a while, especially if tests return numerous errors and everything needs to be fixed and tested again (and again).

In this article, we’ll focus strictly on using dummy emails for testing.

Where to create test email accounts?

Now, let’s go over the most popular services for creating disposable emails for testing.

Mailinator

Mailinator is one of the most popular tools and is focused strictly on workflow testing. It allows for creating multiple inboxes with the @mailinator.com test email domain, which can then be used to test incoming emails. Mailinator doesn’t allow attachments to be added to emails and each account is deleted after a few hours. Their Premium plan allows for API access and 2,000 email reads per day. An Enterprise plan also available.

Price: $159 monthly when billed annually. The very limited free personal plan also available.

Guerilla Mail

Guerilla Mail is a free service offering temporary email addresses without registration. It allows you to both send and receive emails and all incoming messages are displayed directly at the homepage. The page’s UI is rather rough, but you don’t create test emails for aesthetics in any case. With Guerilla Mail, you’ll need to create addresses manually in multiple tabs but for limited testing it might do the job.

Price: free (relies on ads)

Maildrop

Maildrop is an open-source project for generating email addresses in an instant. It comes with a much more pleasant UI than for example Guerilla Mail but the features remain virtually the same. You can pick from one of the suggestions (jealouswhale@maildrop.cc being an example) or type in your own address, for example, “pleaseworkthistimeplease@maildrop.cc”. It also works only with simple text, no attachments. Maildrop also features a spam filter, preventing unwanted messages from entering your inbox.

Price: free (tiny ads might appear)

Tempmail

Tempmail is another tool for generating disposable emails. It allows you to claim random email addresses and checking their inbox without even refreshing the page. As is the case, with all the preceding entries on our list, Tempmail addresses get killed a few hours after creation, giving you space to add completely new emails to your QA process.

Price: free (but heavy on ads)

There are many many other services for creating fake email addresses and with the very same functionalities but it wouldn’t make sense to cover them all here. Bear with us though, as chances are you won’t even need any of these tools.

How to create a fake email address with an existing account?

If omnipresent ads or high pricing discourage you from all these tools, there’s a simple trick you can use. Chances are you have a GMail account that you don’t really use. If not, it only takes a few minutes to create one.

With Gmail you can create unlimited email addresses from within your account. You don’t need to go through any configuration and won’t get any addresses like fnfh3443hj3hgt@gmail.com (unless you request that!). With Gmail, you can simply add a ‘+’ to the username and follow it with any word or combination of alphanumeric characters you can think of. So, for example, for the email address mailtrapsupertesting@gmail.com, we could create:

And so on, the sky’s the limit. Because Gmail treats all these addresses as one, each message sent to these emails will be delivered as if though it was sent from the base mailtrapsupertesting@gmail.com address. Simply add those addresses to your QA procedure and start prompting different processes that will result in an email being sent. If everything is configured properly, each message will end up in mailtrapsupertesting@gmail.com’s mailbox, causing quite a mess but giving you clear information on whether a message was delivered and how it looks for the end user.

To make the data more readable, people usually set up filters. The simplest idea is to add a “signup” label to all the emails arriving to mailtrapsupertesting+signup@gmail.com and do the same for every different address you’re monitoring. It will give you  very clear information on what worked and what didn’t.

This method likely also works with some other email providers but we haven’t tested it yet. If you have a chance, drop us an email (to the real address!) and we’ll be happy to update the article.

Why using temporary emails might not be the best approach?

As you might have guessed from the title of this post, we don’t believe temporary email addresses really solve the problem. They’re perfectly fine if you need to test a very simple web application. By that, we mean a platform where email workflows are only limited to password reset and registration confirmation, without any focus on user permissions, media, formatting of emails and others. Set up a few temporary addresses and you’ll do all the testing within minutes.

The real problem, in our opinion, comes with more complex applications that include but are not limited to:

  • Different user permissions and levels
  • Mobile and web apps
  • Paid and free plans
  • Email onboarding and contextual messages
  • Transactional emails built into the platform etc.

Such workflows require dozens if not hundreds of test email accounts set up. And not only before the product launch but on every single change in the application that could impact email communication. Unless you have a dedicated team at your disposal that can be used solely for setting up and tracking dummy accounts, this immediately disqualifies all these services. By their very nature, they vanish within up to a few hours. If you added 100s of coherenthamster@maildrop.cc (can we please keep this address??) type addresses to your codebase, they would be nothing but useless at your next testing session. The same goes for Mailinator, which allows the creation of hundreds of dummy addresses in seconds. They also disappear within hours and emails sent would obviously bounce on the following tests. What does this mean? That you would need to set things up from scratch for every QA session. Every single time. Save yourself the trouble. 

Another drawback of such a solution is that, without any exceptions, they only support plain text. This might be fine for a password recovery email, after all link and copy are what matters. But if you need to add any formatting, in-line images or attachments, you won’t be able to test them pre-production. Let’s say you wrote a short ebook and put it on your website. Visitors need to leave their contacts and then your site sends them a link to download the book. You got a 10minutemail, tested it and it worked! Full success. But is it really? Could you have missed broken email design, or a misplaced button hiding somewhere on the page. What’s the point of having a working link if users can’t find it?

And finally, there is still a risk of spamming real users with your test messages. Even the most proficient QAs and developers can miss a line of code here and there that handles some email workflow. If the reference to a real database is not replaced with a dummy email address for testing, real users will likely get your “hello world” message. Some might say “hi back!” but most will be rather disappointed with your lack of professionalism.

Can I set things up on my own?

For someone who’s had some DevOps experience in the past, the obvious approach might be to set up the test environment on their own, without relying on 3rd party tools. This would mean:

  1. Setting up an infrastructure to handle hundreds of email addresses on your domain
  2. Generating names and settings for all accounts
  3. Setting up rules for tracking if emails are received
  4. Adding these newly created addresses to your application
  5. Setting up an SMTP server to handle the sending of test emails
  6. Executing the QA tests on your new farm of dummy emails

The biggest hurdle here might be with setting up the SMTP infrastructure, especially if you want to do it on a budget. There are even free, ad-powered SMTP servers available to be used but, very likely, such emails won’t even make it through a basic spam firewall. As a result, you might end up investigating failed deliveries when everything, in fact, is set up properly. To get a reliable SMTP server to set things up, you’ll likely need to spend a few dozen bucks on a monthly basis (+ the regular server expanses).

Even if you do, you will still be facing some of the obstacles mentioned above. If emails are formatting-heavy, you’ll likely need to check each delivery individually. Even if the messages look fine on Gmail, you won’t be sure if they have the same look on Thunderbird or in Outlook, not to mention less popular browsers. If you were to check this manually, this would amount to quite a lot of work you likely can’t afford. Sure, you could integrate your testing environment with other tools to preview emails or check them for spam “eligibility” but this would mean extra time and likely additional subscriptions.

Last but not least, you would still be at risk of sending emails accidentally to real users for the same reasons outlined in the last chapter. 

Is there an alternative to dummy email testing?

Now, if you’re wondering if there’s an easier way to go about all of this, we think there is. Mailtrap was created precisely to overcome such obstacles and make the testing process a breeze rather than a nightmare.

Mailtrap is a software for testing your email workflows without the risk of accidentally reaching your users. It’s a fake SMTP server that can be integrated with your application to perform the needed tests. Since the server is fake, the emails sent don’t reach any real or temporary mailbox. They stay on Mailtrap platform, giving you clear information on what was delivered and how it looked like for end users, regardless of the browser or app they used. Each email can be also previewed, tested for spam and forwarded to real addresses of your team mates. And all of these can be set up within an hour or so and is likely even more cost-effective than your own infrastructure built for the same purposes. Hundreds of thousands of businesses and developers use Mailtrap in their regular QA tests, consider joining this elite group too.

If you have any questions, let us know at support@mailtrap.io.

One thought on “Why Using Dummy Email for Testing Just Doesn’t Work

  1. Pingback: Email Obfuscators and Email Masking for Testing | Mailtrap Blog

Comments are closed.