Product Development Process.

Introduction

This page is an attempt at consolidating and documenting the product development & QA protocols used at Arck . It covers roles within the product development lifecycle, design abstractions like User Stories, how to use them to empathize with users, as well as testing procedures and our processes for validating assumptions.

Divisions of Labor: Product, UX, Dev, and QA

Finding a workflow for a team that results in a regularly-improving product at sustainable costs can be difficult. To that end, we typically divide responsibilities into four categories:

Product defines requirements needed for the business to have a successful product/market fit. It doesn’t equate directly to design, but rather is a model of the business’s product or service offerings – a single source of truth that generates the constraints that UX & Dev create solutions for. They provide feedback advocating for business needs.

User Experience designs & creates the user-facing solution to the problem that Product seeks to solve. They provide feedback advocating for users.

Development designs & creates the technical implementation of the solution that UX come up with. They provide feedback on feasibility & technical constraints.

Quality Assurance validates the solutions that UX and Development come up with against Product’s specifications along with testing any and all assumptions about the product. QA can be many things: automated unit or integration tests, User Acceptance Testing (UAT), manual regression testing, the list goes on endlessly. 

Given the size of our team, these roles are rarely siloed very much – in fact, UX, Dev, and QA will likely overlap frequently, with team members wearing many hats and switching often. 

However, divvying up the work into these categories gives us a tidy way to split concerns, track progress, and measure the effectiveness of it over time to allow us to improve.

Workflow Goals: An Empathetic & Iterative Product Lifecycle

Generally speaking, most software starts out as a solution to someone’s problem; an idea. This person would be the Product person in our model. They identify and define a problem, IE, product requirements, and can either come up with a solution themselves or pass the newfound constraints off to UX to be refined & polished.

It’s worth noting here that it is highly unlikely that the first solution to the problem will be the right solution. Even if the solution is perfect, it’s quite probable that the problem being solved doesn’t always align with the problem in reality. 

Due to this common conundrum, it’s important to make sure that the product life cycle has:

A) Means of validating your assumptions about the problem/solution with people that experience it regularly, and;

B) Iteration, IE, we start with what we have and improve it over and over. This cycle needs to be baked into the product development workflow, as it’s very unlikely the first solution to the first problem will be the best solution to the right problem. 

Integrating an iterative cycle isn’t terribly difficult, but it does require a commitment to testing in some way, shape or form – if you iterate without validating your original assumptions, you don’t know if you’re actually close to a solution to the right problem. So what does a lean & agile startup-friendly lifecycle look like?

1. Define the problem using user stories (described below) to align your problem’s definition with your user’s perspective using feedback and research to ensure you’re aligned w/ your users.
2. Design and create the quickest, simplest, most effective solution to that problem. Proof of concept at most. During later iterations, quick & simplest might be replaced by most-appropriate-for-budget-and-time as a constraint.
3. Test & validate that your problem and solution worked. Measure as much as you can, get as much feedback as you can.
4. Go back to step 1. 

In this way, your product grows and evolves as much as it can, always guided by testing w/ empathy for your users and as much real-world validation as possible, always changing due to new data and ideas. 

Workflow Patterns

One of the interesting quirks of iterative product development cycles is that the same cyclical pattern repeats itself in every stage of work:

Assumption -> Solution -> Implementation -> Validation

This pattern at the highest level is seen in how we break down our roles:

Assumption (Product) -> Solution (UX) -> Implementation (Dev) -> Validation (QA)

However, each one of these roles  has its own internal cycle:

Product iterates as the business grows, learns, and changes as they refine product/market fit.

UX & Dev iterate on their respectful implementations of product requirements. Each iteration, they get a better understanding of the product and being able to provide a solution based on what they learned and executed on the last iteration.

QA iterates on itself to become better at testing, more aligned w/ product, and provide better validation as time goes on. Its job is to grow with the product and always have a way to validate the assumption being tested (or speak up that they don’t).

The above covers the general workflow of our division of labor and the cycles each role goes through, however, there’s still a crucial piece missing – a standardized way of communicating between roles that doesn’t center any specific role, but rather focuses all roles on being aligned with the users of our product.

To that end, there are three concepts we can utilize to keep our product development cycles effective and running smooth: User stories, epics, and acceptance criteria.

User Stories

One of the main ways that we can frame our product’s functionality in an empathetic way is with user stories. User stories are a short description of the problem you’re trying to solve, from the perspective of someone that has that problem. 

User stories  can be remarkably high level or quite granular, but they usually follow a template:

As a {{ type of user }}, I want {{ some goal }} so that {{ some reason }}.

You could also think of a user story as `person x need x purpose`. Here’s a couple examples:

As an unregistered user, I want information on what the product does so that I have a compelling reason to sign up and order a test.

As an existing user, I want a safe way to reset my password so that I won’t lose control of my account if I forget it.

As an admin, I want a quick way to find a customer and mark their order received so that I can process orders as quick as possible.

Epics

User stories are a solid way to understand features, but as your stories get larger and more plentiful,  there’s not much in the way of structure for organizing and managing them. Epics are that structure. Epics are essentially meta user stories – high-level features that can be broken down into many user stories. 

Unlike user stories, epics are much more free-form. While it’s good practice to ground them in a user’s perspective like user stories, epics tend to be more a logical grouping of user stories that are related or interdependent that are released at the same time. 

For example, launching an MVP could be considered an epic. Creating a new page or additional service are also good examples of epics. 

For now, epics are not much more than a bucket to group stories in, but eventually they will become the building blocks for a product roadmap.

Acceptance Criteria

Epics and user stories are a great way to ground a team’s perspective in solving problems for real users, but they alone do not make a bug or feature request actionable. In order to have team members execute on a story, complete acceptance criteria must be defined – it’s the method in which work flows between stages in a product dev cycle.

Acceptance criteria is our shared definition of ‘done’, or how to resolve when a story or epic is complete. It usually is attached to user stories as well as epics. In freelance terms, acceptance criteria is like a SoW document – it’s statement of what needs to be done.

Good acceptance criteria is unambiguous, concise, and as straightforward as possible. Criteria usually comes in three varieties:

• Functional Criteria – Criteria that describes what a story or epic does. Most criteria is this. Example: A user is able to access a list of available reports.
• Non-functional Criteria – Criteria that describes how a story must be completed. Example: A report follows the company Style Guide.
• Performance Criteria – Criteria that describes requirements relating to measurable performance of an implemented solution. Example: Modal fade-out animation must run at a minimum of 30 frames per second. 

Quality Assurance Principles

We now have a division of labor, each with iterative cycles to collaborate on solving problems defined with a focus on relevance and empathy to our product’s users. Currently, of the four roles, we are missing a formal definition and standards for one – QA.

QA can mean a lot of things to a lot of people, so the below principles are an ongoing attempt at a shared definition of how we QA and why.

• Test early and test often. The later you catch a bug in the dev cycle, the more expensive and time consuming it is to correct. 
• An absence of errors does not mean a usable product. Real users should be integrated into the testing process to make sure the product is solving a problem.
• You can’t debug what you can’t reproduce. That doesn’t mean that the issue isn’t there, but its not actionable as anything but a time sink until the issue can be reproduced.
• Perfect testing is the enemy of good testing. There will always be room to improve and unknown factors within the product dev cycle. QA must ensure the most effective testing and product validation given the resources and context of the situation at hand.
• Test cases have diminishing returns. Its common to find a few effective test cases that are easy, but it’s important to expand and diversify test cases beyond those or your testing procedure can miss major issues. This is also known as the ‘pesticide paradox’.

QA Testing & Reporting Procedure

At Arck, we typically use a matrix of user stories as a checklist or which features should be tested when QA’ing an application. While not all changes require testing every feature, all user stories should be tested when performing regression testing after a major change.

For each story, find the associated page/URL to visit it in your testing environment. Note the role and make sure your user account matches it. Attempt to perform the action described (and feel free to ask for help if you don’t understand the action – our QA docs are iterative as well!), and see if the user’s desired resolution occurs. 

When testing stories, it’s important to keep the purpose of the interaction in mind and note any sub-optimal interactions or flows that leave you annoyed or frustrated as a user. What failed you? At what point did the process break down? All of these thoughts are valuable for improving the product and should be logged, even if they aren’t immediately actionable.

When bugs and issues are found, we ask our clients to include the following information, which allows the development team to quickly reproduce and resolve the issue at hand.

• Summary – A description of what the issue is or does. Context for testers and devs.
• Steps to Reproduce – If a bug, you should note the steps required to reproduce the bug. Be detailed, and make sure you can reproduce the bug w/ your steps.
• Expected Results – What should have happened.
• Actual Results – What happened instead.
• Notes – Any other information you think the dev, designer, or tester assigned to the issue would find useful.