End-to-End

Whether you know it or not, you are already testing your application. Whenever you run your app after making a change, you are testing it. For example:

You change the color of a button, you run the app, you see if the button is the new color.
You change the checkout flow, run the app, and see if the checkout flow is the new one.

We call this manual testing and it works well when you're just getting started with a project or feature, but the problem is after a while you realize that it's impossible to ensure your app is working with every change you make. Especially when you make a change that affects multiple parts of the app. In fact, we often affect parts of the app with our changes without realizing it, so even if you were to test every change, you would still miss things.

Whenever we run into manual processes in the tech world, we often find it is more fun (and efficient/profitable) to write software that automates the process. What if instead of clicking around the app to test it, we could write a little script that does this for us. Then when we make a change, we simply run the script and it will tell us if the app is working or not.

I like to think of these scripts as "thousands of little Kent's" running around checking that I didn't break anything. That's the beauty of software development in general.

One challenge with automating any process is ensuring that you are actually automating the process or improving upon it. Normally this isn't too big of a problem, but in the testing world, it's actually a bit of a challenge:

You're using a computer program to make certain that a human can use your app.

You're not trying to make sure that the computer can use your app, but a human. But you're using a computer to do it. You will find yourself fighting against this reality constantly as you write automated tests.

So, whenever you're trying to decide how to write a test, take a step back and think how you would do it if you were giving a list of instructions to a human tester instead of a script to a computer. Remember:

The more your tests resemble the way your software is used, the more confidence they can give you. - @kentcdodds

And confidence is what we're going for when we're writing tests. Also, it's important to know who is using your software (and who is not) so you know how they use it. For more on this, read "Avoid the Test User".

What is a Test?

A test is code that throws an error when something is wrong.

For example:

// here's the thing we want to test 👇
const sum = (a, b) => a + b

// here's the test 👇
const expected = 3
const actual = sum(1, 2)
if (actual !== expected) {
	throw new Error(`The sum was ${actual} but it should have been ${expected}`)
}

That's it. That's a test. Modern testing frameworks do a lot to make writing and running tests easier, but that's the gist of it.

For an in depth look at this, read But really, what is a JavaScript test?

What is End to End Testing?

There are various tools and strategies for testing your application. Each has their own benefits and drawbacks. We're going to start with the End to End testing strategy because it does the best job representing how your app is actually used.

"End to End" is referring to the fact that we're testing the app from the user's experience all the way to our backend components. If you're just getting started with testing, it may feel obvious, but there are lower level tests where you often test individual components or even make fake versions of things. We'll discuss these in more detail later.

Think about the way you're current manually testing your app. An "End to End test" is just doing that with an automation tool.

Playwright

We'll use a testing tool called Playwright to write our tests. Playwright is a browser automation framework that allows you control the browser and interact with it in ways that are very similar to your users.

The Playwright Docs do a good job demonstrating how to get started with Playwright. Most of the time you spend with playwright will be in writing tests, so I suggest you familiarize yourself with the writing tests docs page.

Here's an example from the playwright docs:

import { test, expect } from '@playwright/test'

test('has title', async ({ page }) => {
	await page.goto('https://playwright.dev/')

	// Expect a title "to contain" a substring.
	await expect(page).toHaveTitle(/Playwright/)
})

test('get started link', async ({ page }) => {
	await page.goto('https://playwright.dev/')

	// Click the get started link.
	await page.getByRole('link', { name: 'Get started' }).click()

	// Expects the URL to contain intro.
	await expect(page).toHaveURL(/.*intro/)
})

A few important notes about that test:

page.goto is like the user entering a URL in the address bar and hitting "enter". Because that's annoying, we can skip the full URL and just provide a pathname by configuring a baseURL in the playwright config. With that setup, you can simply do page.goto('/') to go to the root of your app.
page.getByRole is called a locator. It's your primary mechanism for finding elements on the page to interact with. Playwright has several queries/locator functions, and many are based on Testing Library (by me 👋). "Roles" are a built-in feature of the browser used primarily by assistive technologies. Most of the time, the roles are implicit (you're not usually supposed to use the role attribute). By querying using roles, you're also testing (a part of) your application's accessibility to people using these technologies.

Playwright UI

Playwright has a really handy feature called UI Mode which allows you to see what Playwright is doing in the browser and review each step of the test. You'll want to spend a little bit of time getting familiar with the features of UI mode.

Playwright Fixtures

I'm sorry to be the barer of bad news... But we've got another term overload incoming... Here's how I define a "fixture":

A fixed state of the software under test used as a baseline for running test cases.

This often includes database states, configurations, or objects initialized to specific values. It can even include binary files like images or audio/video files.

Here's how playwright defines a "fixture":

Test fixtures are used to establish environment for each test, giving the test everything it needs and nothing else.

It continues:

Test fixtures are isolated between tests. With fixtures, you can group tests based on their meaning, instead of their common setup.

If you squint, we're talking about the same thing, but really what Playwright fixtures feel more like is a way to set up a special environment for a test. Almost like a way to encapsulate the setup and tear down of a test. So it's more of a runtime thing.

Let's look at some code to get an idea of what we're talking about. First of all, playwright ships with a bunch of fixtures by default:

test('this is an example', async ({
	// these are where you get fixtures from
	page, // <-- this is a fixture
	browser, // <-- so is this
}) => {
	// here's my test
})

You can make your own fixtures as well. Honestly, the API is a little funny, and I don't like the way they teach about fixtures in the docs, so I'm going to show you myself.

First it makes sense if you think of it in three phases:

Do stuff before the test runs
Get the stuff the test will use when they use my fixture
Do stuff after the test runs

In the next exercise, Kellie is going to write a fixture for you for the onboarding flow. In that we need to create a user object, then we need to hand that over to the test (with a password), then we need to delete the user after the test runs. Here's how you use it:

test('onboarding with link', async ({ page, getOnboardingData }) => {
	const onboardingData = getOnboardingData()

	// ... do stuff with the test...
})

Here's how that's implemented:

const test = base.extend({
	getOnboardingData: async ({}, use) => {
		const userData = createUser()
		await use(() => {
			const onboardingData = {
				...userData,
				password: faker.internet.password(),
			}
			return onboardingData
		})
		await prisma.user.deleteMany({ where: { username: userData.username } })
	},
})

First off, let me say the {} destructuring syntax is not a mistake. That argument is where you access other fixtures. If you don't need any, you must still include that argument and it must be empty. It's super weird, but you'll get an error otherwise so just go with it.

Ok, so let's break this down:

Do stuff before the test runs:

const userData = createUser()

This is pretty straightforward. Really, whatever you do before calling use is the "before" stuff.

Get the stuff the test will use when they use my fixture:

await use(() => {
	const onboardingData = {
		...userData,
		password: faker.internet.password(),
	}
	return onboardingData
})

This API is kinda funky. But basically, you pass to use whatever you want the value of your fixture to be. So we're passing a function (which is why we call it getOnboardingData), but we could also pass an object or whatever we need.

Like I said, it's kinda funky. But it's a powerful API, so go with it.

Do stuff after the test runs:

await prisma.user
	.delete({ where: { username: userData.username } })
	.catch(() => {})

Once the test is finished, we delete the user that was used to go through the onboarding process. So because this comes after the use call, it's the "after" stuff.

Hopefully that makes sense. You'll be making one in this exercise (for inserting a user in the database directly), so you'll get a chance to try it out yourself.

Also, I did leave off the TypeScript generic, so here's that bit now:

const test = base.extend<{
	getOnboardingData(): {
		username: string
		name: string
		email: string
		password: string
	}
}>({
	getOnboardingData: async ({}, use) => {
		// ...etc
	},
})

The value of the generic is the type of the fixture (whatever you pass to use).