Professional services with an agile mindset

When working with creative or software development companies, transparency is the main ingredient to achieve successful client-developer relationships.

If you currently work for or employ within this industry, I suggest you continue reading.

The main question for outsourced software development endeavors is how can a solution provider dedicate a team for each of their development projects? And secondly, how to create a trustful relationship between clients and developers?

At Scrum Mate, we have seen that it is possible to successfully embed an agile mindset into the relationship between the client and the solution provider. In fact, we are certain that adaptive, agile software development has the potential to deliver more value to the client than traditional, prescriptive models.

In this article, we'll present a model that helps solution providers and clients to build long-term, and mutually fruitful relationships based on the agile mindset.

It's not the execution of the project plan that matters

We don't implement projects for the sake of building products. A product is meant to solve issues, simplify processes and serve an underlying business need.

The first step towards improving efficiency with an agile mindset is to shift our focus from output (i.e. features prescribed by initial concept plan) to outcome (i.e. improvements in business goals). And we want to get more outcome, not more output.

Once we identify the business goals we want to improve - the outcome -, we need to accept that at least to some extent we have no reliable knowledge about how to build a product that meets those business goals.

Here, we find ourselves in front of two sources of uncertainty.

The first one is that we have no experience in building a solution for that specific business need in a particular business environment. In other words, we don't know what to build.

And the second one is that we may have no experience in working together as a client with a developer. In other words, we don't know how to build it together.

Even when we defined the characteristics of the product, we still won't know how it will turn out, or if it will work as we imagined in the conceptualization. We will need to wait until it is used when it's out in the market, and oftentimes, the first experience will be disappointing. At this point, we'll need to keep trying, delivering, and validating product increments frequently and often.

Agile development is all about customers and developers working collaboratively and exploring ways of creating successful solutions together.

Facing high uncertainty, the success of the project will depend on the team's ability to quickly discover and deliver formerly unknown requirements.

The faster we deliver, validate, and adapt, the higher our chance to meet business goals.

Development iterations must be short.

In a learning environment of short iteration cycles, experimenting will yield various options and claims.

Short release cycles and emerging demands for product changes require ruthless prioritization.

Here is the thing.

Real prioritization happens when there are solid and respected limitations of what's possible. Prioritization is about trade-offs.

Prioritization is not about deeming and saying that work is important. But about identifying what you're willing to sacrifice to ensure there are enough resources to focus on that important work.

Constraints are essential ingredients of agile development.

In fact, agile developments use even more constraints than traditional development methods. The difference is that in agile, we know that we often need to drop some of our initial requirements and replace them with new features to maximize the value delivered.

To make good decisions about priorities, clients and developers need to develop a shared understanding about the available options, their costs, expected benefits and the given constraints of the project.

The main problem with the conventional client - solution provider communication methods.

To succeed with agile development, the business project team - client and service provider together - need to deliver and validate working product increments on a frequent basis. A common practice is agreeing on a weekly or bi-weekly release rhythm.

Once agreements are set up, then we can add the secret sauce of agile: The business project team needs to select an appropriate volume of work that fits in the next iteration. That's a limited amount of work for which they will need to prioritize.

Prioritization should be driven by the client. In perspective, clients must have a clear understanding of all the work that should be done to make an educated decision about what to select to implement next. They also need comprehend the limitations of the next iteration.

In contrast to the desired transparency of options and constraints provided to the client, most projects employ a different kind of communication mechanism: ticketing systems. Customer ticketing systems allow customers to send their requests to the developers and attach severity properties to their tickets like "critical", "urgent", "normal", etc. Sometimes they can also set expected due dates.

Those ticketing systems are similar to an opening on a wall.

Customers can toss their requests through the opening or throw them above the wall, but afterwards they have very little information about what happens with their requests on the other side of the wall.

In those "behind the wall" situations, clients lack the transparency about the work that needs to get done and the limiting constraints. Their experience is that they throw their requests through the wall and they receive results in a less than satisfactory speed.

Because clients have no information about what's happening on the "other side" of the wall, they easily get anxious and impatient. This process can result in stricter deadlines and qualifying their requests higher on the criticality scale. This increases the pressure on the development team, creates a sense of urgency and could cause the team to miss quality standards.

If you want to overload and demoralize a team, this is one way to do it.

Stopping this vicious circle can only be achieved by providing full transparency to the clients and transferring the duty of prioritization to the customers.

To improve transparency, it is best to keep a simple list of all the pending work. It should contain details about what the team is working on, and what hasn't been started yet. Yes, what we need is a backlog. The backlog is the common, single source of truth that lists the work that needs to get done, and it's accessible for both the clients and service providers. The order of the items in the backlog defines the order of implementation.

Clients can add new requests to the list and reorganize the backlog based on their current priorities at any time.

The shared backlog allows customers to set realistic expectations by implanting transparency in everything that needs to get done. By having a detailed backlog with priorities set in a disciplined way, we can avoid misunderstandings, such as missed deadlines or flawed planning. As we said before, transparency is an important component of success as it leads to trustful relationship between clients and service providers, which in turn results in better decisions and higher productivity.

How to create viable plans?

Going back to the core of agile development: frequent delivery and evaluation of product increments. Most teams set up a weekly or bi-weekly cadence of releases.

The main question of the planning process is what can fit in the next iteration?

Let's use an analogy to put it in context.

Imagine, that your backlog is a collection of physical papers and models, and implementing them is comparable to delivering them by boat from one side of a lake to the other. The boat is paddled by members of the development team and each iteration is a trip between the two sides of the lake.

The boat has limited capacity to load packages and documents. So, it is important to prioritize what needs to be loaded in the next trip.

To develop a foresightful plan, we can create groups within the backlog to present milestones. We have a rough plan about when, and what features we want to deliver.

At this stage, our schedule is only a wish-list.

To create viable plans, we need a reliable forecasting method and we must determine how much work a team can deliver within the time limits of the selected cadence.

Let's say our desired delivery rhythm is one week. The question is, how much work can a team accomplish within a week, without sacrificing quality standards.

Estimating work

The problem with estimating creative work is that it's impossible to predict how much time it will take to complete the work. Three hours or four and a half? Nobody knows. The only responsible forecast we can provide is saying that a developer can probably complete this work in a half day.

The best way to estimate creative work is using "estimation categories" that represent orders of magnitudes. Using estimation categories speeds up the estimation process and reflects the uncertainty of estimated values.

As an example, here is the estimation scale that our team at Scrum Mate uses.

The first category is the most common development work in our team. It may need around 2-3 hours of uninterrupted, fully concentrated work to implement a backlog item of this size. In reality, it often takes a half day to complete a work like this. We refer to stories of this size as a standard 1-point work. We say that the size of this backlog item is one story point.

The second category is about twice as big as the first one. It's a 2-points sized work. It requires 4-6 hours of uninterrupted, fully concentrated work to implement a backlog item of this size. In reality it usually takes almost a day. Sometimes less, sometimes slightly more.

If we continue going up, a backlog item that is estimated to 5 points may require 10 - 15 hours of fully concentrated work. To implement a story of this size may take usually two, sometimes close to three days.

More than 90% of the work our team starts working on fits into one of the categories between one to five points. The rest is smaller than one point, meaning that it takes around an hour or less to complete it.

We try our best to avoid working on stories that are 8 points or larger. We decompose those backlog items into smaller chunks before starting to implement them. A story in the 8-points category can engage a developer for a whole week. They are too big to oversee their complexity and can cause unpleasant surprises.

You probably noticed that we define required effort in ideal working hours, that is uninterrupted and fully concentrated time spent on a task. No star-gazing, no questions, no problems - just effective work. The equivalent of 6 hours of fully concentrated, uninterrupted, ideal work is around one workday.

We're not ashamed to admit that we have only a rough idea about how much time it will actually take to complete a backlog item.

However, with the estimation categories described above and the forecasting mechanism that we will explain next, we can anticipate fairly well how many backlog items we are able to implement in the next week or two.

Creating forecasts from estimates

The best way to forecast the future is to learn from the past.

For example, we may see that in the past, the team was able to deliver 50-60 story points per week. Knowing this, we can assume that the team will be able to deliver the same amount of work in the future. This is what we can identify as the velocity of the team.

Going back to our boat analogy, if we load an estimated 50 story points of work, then we can deduce that team can ship it within one week. While if we try to ship 80 story points, it will take more time to reach the other side of the lake.

We need to be careful with forecasts and recognize the capacity limitations of the team. Just like the boat has its load capacity, try to put too much work on the boat, we'll exceed its bearing capacity and it'll become iffy and slow, it could even sink.

Once we have enough data from the past, we can see that the team's velocity varies within a certain range. And still then, what happens in reality depends on various factors: the complexity of the work, a less than ideal environment (more interruptions than usual, there was a flu epidemic) or some of the team member's individual performance was lower than usual.

Those factors altogether can easily result in a +/- 25% variance in the team's actual velocity on the actual iteration.

The variance in the team's velocity is unavoidable, and it should be clear for the business project team.

Agile team working on many projects

Agile frameworks suggest that a team should be dedicated to a single project, which we think is wise advice.

However, dedicating teams to a single project is not feasible for professional creative services companies. If we dedicated a unique team for every client or project, we would lose our ability to adapt to the changing demands of our clients. A team dedicated to a single client would be either overloaded or underutilized most of the time.

Teams of agile creative service providers need to cope with more than one project and client at a time. This is the only way we can maximize the potential of the company.

Even if a team works for many clients, there are a few principles we need to comply with to obtain the benefits of agile.

Agile teams are self-organizing, meaning that team members have all the knowledge and authority to decide how to get the work done. Nobody from outside the team has the same level of understanding about the problem, so no one outside the team has the authority to command team members. It doesn't come without responsibility, team members are accountable for each other and this mutual accountability drives agile teams forward.

To enable self-organization, all team members must have all the knowledge to make decisions and hold accountability. Transparency is provided by the team board that contains every backlog item the team is working on.

To support sustainable performance, it's crucial to maintain the balance between workload and capacity.

There is a bullet-proof way to ruin the productivity of a team: long-term overexertion.

Unrealistic deadlines and impractical volumes of workload lead to burn-out, low quality products, and defects which generate critical issues - a vicios cycle so to speak.

When a team serves many clients, they will face an extreme pressure caused by the continuous stream of requests from clients. We need to manage this pressure to avoid overloading the team.

The best way to manage pressure is to switch from a push model to a pull model, where the team can pull work from the prioritized backlogs prepared by clients and account managers to their board.

Agile teams don't need managers to select the work they can presumably deliver during the next iteration. They need clear priorities and guiding principles for selection. According to the 5th agile principle, we need to build projects around motivated individuals and give them the environment and support they need, and trust them to get the job done.

This is how you can implement the pull model with an agile team and many clients.

Create an intermediate area, where every client (or their account managers) can place the backlog items they want to implement next. This intermediate area serves as a lock between the team's board and the flood of requirements from clients.

The advantage of this lock area is that it presents an already filtered amount of work for the team, so they need to deal with a manageable volume of tasks when pulling work to their boards. Also, teams and clients (via their account managers) can agree on the limits of the intermediate area. Let's say they can agree that every client can add up to 20 story points work as not to overload the boat .

In our agile team collaboration tool designed for teams working with multiple clients, we call this lock area the 'Planbox'. Business stakeholders (e.g. clients and account managers) can add their most urgent requests to the 'Planbox'. The team can then access the unified view of the 'Planbox' sections of all connected projects and pull work directly to their board.

To sum up

Building a customer - service provider relationship upon an agile mindset has vital advantages for both parties.

To make it work, there are only a few principles to comply with.

Here they are.

Let's give up the innervation that the goal of the project is to produce an envisioned product. The goal is to improve the business. The product envisioned upfront is a potential solution to improve the desired business metrics, but it's potentially flawed.

Set up a business project team from the representatives of the customer and solution provider and define the constraints within the project team aims to deliver the anticipated improvements in the selected business metrics.

The likelihood of success will depend on the project team's capability to explore the characteristics of the successful product and the project team's ability to learn how to build that product in cooperation. Both require frequent delivery and evaluation of working product increments.

Frequent delivery and evaluation of working product increments will open the way for countless new ideas. The required effort to implement all the initial and emerged ideas will far exceed the boundaries of the project. The project team needs to carefully select what features to implement, what to postpone for later iterations and what to give up entirely.

The success of the development will depend on effective prioritization rather than on the implementation of the original project plan.

Prioritization means ruthless selection and painful trade-offs, but without setting priorities, the team would waste their resources on developing nonessential product features.

The right tool to aid prioritization is a backlog. The backlog is an ordered list of everything that is known to be needed to accomplish our goal. The backlog should be transparent and available for both the client and the service provider.

Agile solution providers employ self-organizing development teams. Unlike product development teams, service provider teams are working for more than one client, so they are not dedicated to a single project. This is to maintain their ability to adapt to the varying volume of work arriving from a customer.

Members of an agile development team need up-to-date information about everything that needs to get done and information about who is working on what. This transparency enables the self-organization of the agile team and it is achieved by representing all work that needs to get done in a single, visible board. The development team usually works on tasks from more than one client in parallel.

The development team owns the implementation process, and the business project team owns the identification, decomposition and prioritization process. In other words, the business project team owns the backlog, the development team owns the board. To make the cooperation smooth between the development team and the business project team, there must be at least one person who is a member of both teams at the same time.

To shelter development teams from work overload, agile service providers need to employ a pull system controlled by the development team. That is, business project teams can't push their requests to the development team, but they can place their issues to an interim area where the development team pulls work to their board. This interim area is called the 'Planbox' in Scrum Mate.

Clients and service providers should agree on the maximum amount of work allowed in the 'Planbox' and a service level for the average time within the team will deliver work placed on the 'Planbox'.