Accountability in a matrix environment

In the beginning, was the Boss

It’s hard to believe, but in the olden days, before self-organizing teams and servant-leaders, everyone had a boss. Organizations were managed vertically, with teams in strict hierarchies. Bosses told you what to do, how to do it and when to do it. If you didn’t do it fast or well enough, your boss would discipline or perhaps fire you.

Management by diktat doesn’t generally lead to desirable outcomes, so many organizations spend time analyzing strategy, breaking it down into progressively smaller tasks that can be ‘cascaded’ to underlings (to use the jargon). Everyone is given objectives that measurably contribute to the overall strategy. At each level, individuals are accountable for all the activities of those that report directly or indirectly to them.

While this approach gives context to objectives, they are still instructions from above. This works well when people are collaborating on a highly planned project, such as conducting brain surgery, building a pharaoh’s pyramid or invading a small country. And, if you want to be able to concentrate blame on an individual so you can fire them when things go wrong, it’s perfect!

In creative endeavors (e.g. making software), this doesn’t work well because whipping rarely improves the performance of people whose motivation is to make great stuff. If you manage people with a diverse range of skills, as in a typical product team, it’s also challenging to understand and assess performance, thus harder to coach them.

Furthermore, there is little opportunity for people at the lower reaches of the hierarchy to challenge assumptions made several levels above them. There may be continuous improvement processes (e.g. Kaizen) in place, but they’re designed to remove waste and blockages, rather than to enable delivery teams to take ownership on a day-to-day basis.

We need to talk about Erica

Erica worked at Fox Lab technology, an agency that built bespoke business software. A capable engineer who enjoyed working in a team, Erica wasn’t a clock-watcher, but neither would she throw herself onto a pyre for the company. Fox Lab had a vertical management hierarchy, and Erica found the managers to be affable, albeit usually quite stressed. The directors were distant and, despite working there for two years, she’d never spoken to the CEO, Carole.

Working at Fox Lab was a series of broken deadlines. Instructions were passed down without context or rationale. Erica’s manager, Martin, would tell her to work on this, then drop it to work on that, these pivots often occurring immediately after he’d been in one of those shouty management meetings. Martin had a cordial relationship with his team but often had an exasperated tone, scarcely concealing his frustration. He had risen to his position through dint of excellence as an engineer, but a few years in management’s blame zone had left him tight-lipped and resigned.

Morale was rock-bottom, and the only thing keeping people at Fox Lab was their generous salaries – “blood money” they termed it. You would never have guessed this from its external messaging. It worked very hard on PR and, despite its hollow soul, it occupied a strong position in the market. Clients were frequently rankled by its serial failure to keep promises, but one of its execs was an expert spin doctor who always managed to schmooze them. This usually meant making more unsafe promises based on assumptions that hadn’t been sense-checked with the delivery team.

In truth, Fox Lab was a ‘zombie’ company – unaware that it was already dead, and only a few customer desertions away from a rapid demise. Carole the CEO never let her team know about her sleepless nights due to constant anxiety… why couldn’t she seem to get things turned around? Why were they always on the back foot, continually missing deadlines and disappointing clients, despite her efforts to realign expectations? She sought and gained assurances from her directors, but time and again, these proved worthless. She avoided blame culture, but nobody ever seemed to be accountable and she wondered why nobody had been disciplined, let alone fired! It was always just excuses – different excuses every time but always rooted in someone claiming they hadn’t been informed, something had been oversold or someone had been promised resources that hadn’t been forthcoming.

How did it come to this?

In Fox Lab’s early days, everyone knew what everyone else was doing, and they mucked in to help each other. As the company grew, it became necessary to appoint managers to support and keep tabs on people. Carole increasingly found herself spending time with large clients, persuading them to invest more in, rather than ditch Fox Lab’s services. She promoted some managers to directors so they could manage the company day-to-day. They, in turn, promoted the more trustworthy amongst their teams to manage the teams on their behalf while they strategized.

Every now and again it would become obvious that there were too many chiefs and not enough Indians, or that a particular department had grown too large or small, and a restructure would ensue. Directors would do some more shouty meetings, boxes and arrows would be drawn, and announcements made. The teams would sigh, spend a few months getting used to the new arrangements (or quit), then carry on carrying on. They were used to these false dawns, with the periodic arrival and inevitable departure of heroes who were going to fix everything with their new way of doing things.

In all this continual strategic realignment and hardening of management attitude, nobody had considered what Erica could do to help. Sure, her manager had assessed her engineering abilities, and she had been given appropriate KPIs and targets to reach, but the only thing Erica had control over was lines of code. She had read about the Hoshin Kanri management philosophy employed at Toyota and considered her profession to be Genchi Genbutsu (the stuff that really matters). Depressingly, however, she spent her days making things that nobody would ever use, driven as they were by corporate angst rather than real customer need. She was well paid, but she longed to be able to express herself and go home feeling good every day. Once she’d saved enough for a mortgage deposit, you wouldn’t see her for dust…

Sh*t rolls downhill

Fox Lab’s problems are caused by its top-down management philosophy, in which teams are given commands with deadlines. Carole isn’t a bully, but like many CEOs and founders, she is driven and multi-skilled and had grown accustomed to making calls without challenge. She didn’t deliberately set up a pyramid of control, it just happened by default. Even in modern businesses, most people want someone else to make decisions for them. As is generally the case for a social species, this tends to create control structures with a ‘pecking order’, in which the most assertive rise to positions of power.

This is fine when you’re a small team, but as the business grows, even the most energetic and curious manager cannot track and understand everything happening under their watch. The result is that when things go wrong – which happens in any creative industry – managers struggle to identify or address the causes of failure. They’re held accountable for something they cannot actually control, and their only recourse is to apply more pressure downwards whilst massaging information flow upwards. This ‘boss problem’ can lead to a toxic macho culture with bullying, cover-ups, and even corruption. So, if the vertical command doesn’t work, what do you do?

Enter the Matrix

In the 1970s, Matrix Management emerged in response to the increasing complexity in global businesses, which brought conflicting demands from product, functional and geographic divisions. It provided the flexibility required to keep up with the accelerating pace of business change and competition and combatted oversimplification, silos, and parochialism within the business. The concept introduced a multi-dimensional accountability structure, with both vertical and horizontal accountability, easing the deployment of people to new initiatives. Project output was separated from personal development, and everybody had two ‘bosses’ – your line manager who would look after your welfare, performance and professional development, and a ‘dotted line’ report to your project manager who would direct your project-specific activities.

This is termed ‘horizontal management’ due to its cross-organizational way of working, recruiting staff onto projects from vertical functional units. The project manager’s decision-making power varies – in ‘hard’ implementations, they rule supreme; whereas in ‘soft,’ they depend upon departmental heads for authority. Larger organizations may further increase flexibility by creating a ‘bench’ system, in which staff who aren’t currently employed on projects wait for a suitable project to come along. They may be deployed to projects or apply to work on them voluntarily.

Most of us work in matrix environments, officially or otherwise. It’s rare for an organization, especially in the technology industry, to operate purely on a hierarchical basis. Capable and proactive people don’t thrive when confined by artificial boundaries and tend to be ‘lent’ to other teams or help them out voluntarily. If this ‘horizontal pressure’ isn’t released it leads to frustration amongst high performers – quality will out, and if you don’t let talented people express themselves, then they will find somewhere else that will. An indicator of this problem is serial restructuring, in which the company keeps reorganizing its teams to enable the right people to work on the right things, resulting in loss of momentum and confidence in management. Organizations like this don’t know they’re in the matrix, but they can tell there’s something wrong with the world.

I’ve worked in the software industry for over two decades, and in most of the roles: development, support, consultancy, operations, pre-sales, sales, marketing, management, and strategy. In all that time, I’ve rarely met anyone who thought that matrix management was easy, and many that thought it can’t work. Critics cite a lack of accountability, priority conflicts, the difficulty of reporting and even people gaming the system. In one case I knew of, someone ‘working’ at a large systems integrator spent eighteen months on the bench, on full pay, before he was finally rumbled!

How to entirely miss the point of matrix?

Trying to fix the boss problem by giving everyone another boss isn’t likely to succeed. It causes ambiguity of purpose and accountability. How do I prioritize? How can I be held accountable for something I can’t control? Who evaluates performance? What is performance if it’s not doing what you’re told to do? Faced with conflicting pressures, people will tend to align with a tribe in a turf war, playing bosses off against each other and descending into a perpetual cycle of blame/excuses or even just simply giving up. Warning signals include conflicting communications, a proliferation of meetings and reports. Trust and accountability break down, leading to high stress, absence, and churn.

Just as overlaps are created, so too are voids into which proactive people naturally jump, without a mandate or clear accountability. They end up over-reaching, causing as much trouble as they solve, creating tensions in teams and burning themselves out. Other high performers are tactically shuffled between projects, with a consequent drop in productivity due to spin-up time and disruption caused to established teams. Top-down has been replaced by a pincer movement, top-down and from the flank.

Unfortunately, there’s no shortcut to a culture of empowerment and accountability, as put by Christopher A. Bartlett and Sumantra Ghoshal in the Harvard Business Review (July/August 1990):

“…companies that fell into the organizational trap assumed that changing their formal structure (anatomy) would force changes in interpersonal relationships and decision processes (physiology), which in turn would reshape the individual attitudes and actions of managers (psychology). But as many companies have discovered, reconfiguring the formal structure is a blunt and sometimes brutal instrument of change.”

Customer value is determined by customers, not bosses

To be accountable for something, one must be able to influence its outcome. Even in pull systems like Scrum, where the delivery team chooses what to deliver in each timeframe, if they’re doing so under pressure from a boss, then it’s unlikely they’re being driven by customer value. They will constantly be trying to speed up the things the boss wants to be done. Even if that boss is a Product Manager who has good data or a knack for spotting customer value, the intellectual and emotional constraints they place on the team will distort their behavior. Moreover, the team cannot be truly accountable for their output, because they never had any choice about what they did. This leads to learned helplessness, manifested as ‘cargo cult’ or ‘death march’ projects.

To enable the team to perform to their potential, they should be seen as a microcosm of the company, embodying the corporate culture and focus on customer value – your company has that, right? Much has been made of the role of Product Manager as representing the voice of the customer but consider how effective it can be if they truly are the customer of the delivery team. By implementing an internal economy in which horizontal relationships are based on customer-supplier relationships rather than executive control, you can focus people’s attention on their responsibilities to each other rather than what they’ve been told to do by an individual.

Performance is agreed and enabled, not forced

A criticism of matrix is that it requires really capable people; I’d say it’s the only way you can attract and retain those people and build a culture of high performance. Talent won’t tolerate decision paralysis, a sort of ‘corporate yips’ caused by a chronic lack of strategy and/or management information. They just want to get on with the job, and the internal economy places trust in all team members to make the right decisions.

“The reorganizations that work best don’t just reshuffle the boxes and lines on an org chart. Rather they improve a company’s ability to handle its most important decisions. They enable people in the organization to make better decisions. They speed up decision making. They also increase the ‘yield’, or the proportion of decisions that are executed effectively.”
– Decide and Deliver, Bain and Company 2010

Traditional team architecture is defined by bosses, job descriptions and KPIs. There may also be a RACI, er, matrix detailing everything for which you are Responsible or Accountable, or to be Consulted or Informed. However, RACI often incentivizes people to do anything other than creating value (see Goodhart’s Law, or “Whatever you measure, that’s what you get”). It also doesn’t answer the question “What needs to change?” Simply saying that an engineer is “responsible” for the launch of a product doesn’t help much – arguably the office cleaner is also in some way responsible. RACI is useful only for singling someone out to blame.

To meaningfully track a role’s performance and development, it’s necessary not only to define what is to be done but why and for whom. Just as any product has its target market and value proposition, so it’s vital to focus on the value that each role offers. This need not require a full product strategy – imagine trying to run a business in which every role was itself a business – but focusing on outcomes will ensure that people aim to deliver value and can be managed by exception. One way of defining this is in a simple, um, matrix:

At first glance, this may look complicated, but it’s no more so than writing someone’s job description with structure and context. The term “KPI” (there I said it) has become something of a swear-word in the tech world, perhaps unfairly – there’s nothing wrong with measuring things, the mistake is to measure the wrong things or with a lack of precision. In a matrix economy, defining success metrics for each ‘deliverable’ from each role to each other takes the form of a contract between those roles; these contracts can be negotiated by the team so that nobody can complain that unfair requirements have been pushed onto them.

For example, traditionally, you might specify that an engineer should “write code using Javascript” and put them through exams. This will certainly measure their coding skills but won’t tell you anything about their ability to work in a team, focus on outcomes or take accountability. Instead, you could require that they “create product increments” with the acceptance criteria being that they are stable, secure, performant and written in Javascript, with performance metrics agreed by the team.

Embracing the matrix economy

It is not a simple or easy journey to move from top-down, vertical command-and-control to the matrix economy model, with its focus on outcomes rather than compliance, its requirement for a high degree of trust and the shift from management to coaching. The whole organization needs to buy in, starting at the top, and it will take time to make the transition.

The job of the line manager will no longer be to tell the team what to do, but to coach them, ensuring that they are equipped to succeed and insisting that they give it their best. The traditional concept of a Project Manager that directs everything about the project doesn’t fit well into this model, because the ‘customer’ of the team should be concerned only with the outcomes from the product development and not how it was done. Thus, Product Managers focused on customer value, are needed instead. I’d define a maturity model but as Product Focus produces their own, I’ll point you to that one instead.

Negotiating between individuals, teams, and functions in this way leaves no place to hide when it comes to accountability. The performance or procedural issues, within and between teams, can be identified unambiguously because they’re reckoned against specific and measurable duties. These, in turn, drive customer value and thereby business success. Time to take the red pill.

Aidan Dunphy
Product Focus Senior Consultant