A Bug (Hunter’s) Life: Quality Assurance at ConvertKit

Hi, I’m Katrina, and my manager once called me “a one-person QA army.”

I don’t know anything about an army, but I’m the only person at ConvertKit with quality assurance in their title (it’s “Senior Quality Engineer” if you want to get technical), so I guess we’ll run with it.

What’s it like to be the only official QA supporting dozens of people across multiple teams? It’s tough in some ways – like I said, there are dozens of them and only one of me! – and pretty effortless in others. Our teams are incredibly considerate about my workload and always make sure I have bandwidth to take on a new task before they add something to my queue.

My to-do list on any given day might look something like this:

• Test a new workflow that will help creators earn a living more easily
• Take a look at behind-the-scenes changes that make the Broadcast editor better
• Make sure tweaks to a slow query don’t have knock-on effects anywhere else in ConvertKit
• Meetings to stay in touch with the engineers and learn what’s coming up for me

That last item doesn’t seem very exciting or interesting, but it’s the most important part of my day. As you might have guessed, I get pulled in a lot of directions at work. Setting priorities across teams intelligently with the help of other people, I can use my time efficiently and spend as much time as possible doing my favorite part of the job: hunting bugs.

I get plenty of assistance in that process: product managers help me understand the business need that’s driving the work, which gives me insight into how creators might use the feature; engineering managers can help me understand any technical difficulties that popped up during development, which gives me leads on parts of the work that might require extra attention; and engineers write testing steps on their pull requests, which shows me the happy path through the work and gives me ideas for places to test out what happens when I don’t follow the happy path.

But I’m not the only person at ConvertKit who tests the product. “Quality” is in my job title, but it’s everyone’s business, every day. I can’t possibly review all the great work our engineers do; after all, there are only eight hours in the workday, and my DeLorean is in the shop. Instead, some things go through “dev qa,” when an engineer puts on their detective hat and investigates a colleague’s output for bugs. That keeps everyone’s wits sharp and the code in tip-top shape.

I hadn’t thought much about it until now, but this process reflects so many of ConvertKit’s values.

Teach everything you know

Every day of the week, someone teaches me something new about how ConvertKit works, and I get to teach my colleagues new QA tricks to uncover bugs.

Create every day

Engineering at ConvertKit isn’t just a job - it’s a creative endeavor that produces a tool that helps creators earn a living online. Collaborating on a project like that is a challenge I savor.

Work in public

I give engineers feedback on their work in comments on pull requests where everyone can see and (hopefully!) learn from them.

Default to generosity

I think this is the real secret sauce. There are teams out there in the world where QA and engineering are on opposite teams: QA “breaks” the software, and the engineers grumble about it. Defaulting to generosity means no one at ConvertKit has ever accused me of breaking anything. Instead, we put our failures behind us and work together to make something even better.

So QA at ConvertKit is a job for a one-person army, but it turns out I’m not alone on the bug battlefield after all.