Thursday, March 31, 2016

Software Development Process in a Scaling Organization

DreamBox is growing. As we grow, we are learning to be more adaptive and ensure efficient scaling as things change. In Engineering, we formed several teams to build our solutions. Having several independent teams has given us the ability to parallelize much of the work we do, and move faster to serve our customers. As a result we had to improve our ability to synchronize and coordinate across teams, and create some additional structure around the daily engineering operations and the development cycle.

We had already been using the roles of Product Owner (PO) and Scrum Master (SM) extensively for years. Those are traditional Agile Development roles, and their clear definition worked well for us. In addition, we felt the need to formalize a third role in each team: the Project Lead (PL). This is nothing new as this role usually exists in development teams in one form or another, but we found it useful to formalize the PL as a specific person for each of the scrum teams. The formalization clarified roles, improved communication, and prevented running in circles when it came to organizing the process to keep the teams synchronized and focused on common business goals.

Together these roles represent the three legs of the stool that facilitate, support, and guide the teams to work in an organized and efficient way. Effort with positive impact can best be achieved by ensuring accountability and ownership of strong customer, product, business, and technical vision to guide the scum teams to efficiently meet the right goals.

When defining any kind of role it is critical that everyone involved is on the same page about the need for them, who is filling them, and that the company is constantly learning and adapting to changes and needs.

In this chart the three roles are represented as circles, and the areas of interaction are labeled with letters.

The chart shows how the three roles:
  1. Operate independently, performing their specialized functions (A,B and C).
  2. Interface, understand and support each other, and help each other make the project successful (D,E and F).
  3. Collaborate tightly as a team to ensure the delivery team has clear priorities, technical vision, and supporting structure and process (X).
The circles are positioned horizontally based on their level of focus. External focus (customers) on the right, and internal focus (the company and the process) on the left. The PO is mostly focused externally, but is constantly interacting with the team. The SM is mostly focused internally, and is all about making sure the team is not impeded. The PL is in the middle, and needs to understand both focus areas, but not necessarily in great depth.

The circles are also positioned vertically based on the amount of detail they need to worry about and keep in mind. The PO stays high level, but has an ability to dig in and understand the details when the team needs them to. The SM is slightly lower level being aware of the day-to-day details, but does not need to know everything about the technical implementation. The PL needs to know all the technical details of the project, but also have an understanding of the business vision, the customer needs, and the overall process.

At DreamBox, responsibilities of these roles are described as follows:

Product Owner

  • A Product Owner (PO) is the empowered single point of product leadership. A PO should simultaneously represent internal and external views (understanding the needs of the organization and stakeholders, customers, etc.) and clearly communicate to the development team what is needed and in what order to build it. A PO's role includes: 
  • Managing the economic decisions of what is being built (business value, cost/benefit, opportunity cost). 
  • Participating as a key player in planning, providing valuable input which enables the team to select product backlog items to commit to for delivery in a sprint. 
  • Grooming the backlog, which includes creating, refining, and prioritizing. 
  • Defining the acceptance criteria for stories so that teams can ensure that what is being built is in fact what the PO is asking for. 
  • Collaboration with the development team on a regular basis, keeping an open feedback loop throughout the development process. 
  • Collaboration with stakeholders as they are viewed as the single voice for all of the stakeholder community using this input to inform decisions about the product backlog.


A ScrumMaster should be viewed as: 
  • Coach to the PO and delivery team.
  • Servant-leader to help the team become efficient and continually improve.
  • Process authority for the delivery team, not in setting the process for the team but rather in helping each team establish the process that works best for them. 
  • Shield the team from interference and interruption so they can focus on their committed sprint. 
  • Impediment remover, helping the team maintain productivity.
  • Change agent to help evolve agile practices in the organization. 

Project Lead

A Project Lead (PL) s a member of a team that is responsible for:
  • Always being informed about all technical aspects of the project.
  • Expected to clearly articulate how the implementation achieves the project's vision and goals to the team, other teams, and stakeholders.
  • Provides communication and collaboration about the project with other teams.
  • Proactively understands, shares, and addresses cross-team project dependencies.
  • Understanding and socializing implications of project decisions on customers and client-facing teams (Sales, Marketing, Finance, Customer Support, etc.) and creating opportunities for team members to get close to the customer.
  • Drives the technical vision, strategy, tactics, and direction of the project.

When Things Don't Go Well

These three roles are very important. If one of them is not fulfilled, symptoms of dysfunction start cropping up. Here we examine the various issues and symptoms of not effectively carrying out PO, SM, and PL functions.

When the Product Owner is Not Effectively Engaged

The PO carries the business vision of what the team is building, and it is absolutely necessary to have an engaged PO for the team to succeed. If the product owner is not effectively engaged with the PL and the SM, one or more of the following symptoms may be observed:

  • The backlog gets too short, and the team doesn’t know what’s coming next.
  • General lack of priority. The team doesn’t know what is most important, and due to the lack of vision, they start making up priorities. Often they take on work items such as experimental stuff to fill time (there is nothing wrong with experimentation, but experimentation needs to be deliberate, not accidental).
  • The PL and/or the SM have to start collecting information to make business-priority decisions. That slows them down from their other important functions, and the decisions made are often not well informed.
  • There is no product vision or business vision. Engineers end up making all the product decisions based on insufficient information or wrong assumptions.
  • Poor product vision creates a lack of continuity in the user experience. Products ends up looking like collections of features, not integrated solutions.
  • Stakeholders, and the business as a whole, don't know what's going on with the projects: (1) What is being released, (2) What is being worked on, (3) What's coming, (4) When things are coming.
  • Since there is not a clear vision, stakeholders get nervous and start creating fire drills, throwing work items to the teams directly, without a clear indication of relative priority. Without a PO helping to negotiate priorities with the stakeholders, the team starts reacting, jumping on the latest fire drill immediately, interrupting whatever they were already doing. Very little ends up getting completely done this way.
  • Since lots of work is started but not finished, we see lots of effort without much business impact.

When the Scrum Master is Not Effectively Engaged

The SM facilitates the inner workings of the team, and ensures that process or other impediments do not get in the way. If the SM is not effectively engaged with PO and the PL, we risk one or more of the following symptoms:

The team becomes disorganized.
  • The team spends a lot of time discussing process or simply ignoring the process.
  • Since the SM is the guide through an organized process, without an effective SM the process seems to get in the way of progress instead of facilitating progress.
  • The team becomes unpredictable – it is unclear to the PO how much time anything will take to finish.
  • It is difficult to measure the effectiveness of the team and the status of a project.
  • It is hard for the PL to keep track of what's happening and who is working on what.
  • The technical vision and the business vision start drifting from the actual work that is being done.
  • Impediments do not surface, or surface too late, blocking the team members for prolonged periods of time.
  • The team spends the majority of the time on un-prioritized or tracked work.

When the Project Lead is Not Effectively Engaged

A PL carries the technical vision of the project and/or can explain what's going on with the project in great detail. If the PL is not effectively engaged with the PO and the SM, we risk one or more of the following symptoms:

  • General lack of technical vision. 
  • The team often has to ask the question: Where are we going with this?
  • Engineers start making incompatible decisions in the team and across teams.
  • There is no effective cross-team communication. 
  • Different teams start stepping on each other's toes, especially at release time.
  • Technical roadblocks are often found at integration time.
  • Various scrum teams start developing different and incompatible jargon and terms. 
  • Team is unable to settle technical disagreements.
  • Team is often surprised by technical choices made by other members of the team, or by other teams.
  • SM and PO are overwhelmed by technical details that they don't necessarily fully grasp.
  • Nobody can explain how things actually work.
  • Nobody spots (early enough) product designs and visions incompatible with the technical reality.

Written by:
Lorenzo Pasqualis, Sr. Director of Engineering at DreamBox Learning
Eryn Doerffler, Senior Project Manager and Scrum Master at DreamBox Learning