Do you like what you are reading? Do you want to receive more content like this, in your inbox?
I send out articles just like this to my newsletter about once per week. Subscribe now:
* indicates required
Do you like what you are reading? Do you know people who might be interested too?

Share This Page:

Maybe You don't Hate Agile - Maybe You Just Need Some Coaching

Some days ago I tweetet...

Why do developers hate Agile? Allan Kelly has a theory, and according to him, it mostly boils down to three reasons:

  1. Agile is now often an imposed change - The desire to become agile does not come from the developers, it is imposed on them by management.
  2. Agile without technical quality/skills makes developers lives worse.
  3. In the wrong hands, the same agile tools are very effective micro management tools.

Today I want to focus on the second and third reason. This does not mean that #1 is not interesting or problematic - quite the opposite! But I will leave this problem alone for now, because it can solve itself over time when you get #2 and #3 right.

Also, the second and third problem fit very well into another discussion I am involved in right now:

Agile Coaching and Technical Coaching

In the last few months, I had discussions with multiple people. We talked about how companies that get agile coaching also need technical coaching, because "Agile" without technical skills is meaningless (#2 of Allan's reasons immediately reminded me of this discussion).

The discussion is, for now, mostly about how we can create awareness for agile AND technical coaching in companies and how we can work together as different types of coaches. I also wrote a blog post about it: The Missing Link in Agile Coaching.

But, let's get back to problems #2 and #3 now...

Tools for Micro Management

The same tools that can help a team work together effectively - Like visualizing all work and problems, having regular planning meetings, sprint retrospectives, daily standup meetings, ... - can also be used by managers to micro-manage a development team.

Then they do not only lose their purpose - They become dangerous. Team members will be less and less motivated. The best will quit first. Some will stay but "tune out" - They will only do what is absolutely necessary to get a pay check. They will write angry blog posts about why they hate Agile. Hiring new developers becomes more difficult, because the company will have a certain reputations.

As a company, you do not want to get into a situation like that!

I also realize that there is a bigger problem here: That this abuse of the tools is even possible. But I will not even try to solve this right now. Anyway, I would love to see more discussion about it ;)

Lack of Technical Skills

Allan's idea here is that, without technical excellence, working in an agile way actually makes things worse for developers.

When you do Scrum, for example, you must have "potentially shippable software" at the end of every Sprint. And "potentially shippable" means that you can deploy it to production at a moment's notice, without (much) extra work. And you should have deployable software after every commit, since every commit will be deployed to a staging environment (or even to production!).

Working in an agile way means releasing to real customers in very short intervals - We want go get real feedback, really fast. It means changing direction often, and changing requirements, because we will know more about the problem and possible solutions once we get feedback.

To achieve this, you need a very high quality, good testing practices, good - executable! - specifications, and a software architecture and software design that is easy to change. If you do not have those things, realeasing often and deploying every commit will be quite painful...

Let's assume that (almost) everybody actually wants to do a good job. They want to get better at what they do. How can it happen that entire teams lack the technical skills they need to make agile software development work for them?

The Relationship Between the Two Problems

Many teams who lack technical skills also have the other problem - That management is using agile methodologies "against them".

But I think they do not do it on purpose - Managers, too, want to do a good job. Sometimes, not everybody fully understands how they are supposed to use the agile tools. Sometimes there is a lack of trust: Some managers have been burned before and are now extra cautious. If your company has a KPI/bonus system, it probably incentivizes bad behaviour - your company is actively destroying value by having bonuses!

So, if management is "using the agile tools against the team", the team probably also has no time to learn, experiment and grow. There is no conference budget. There is no training budget. The company often has processes and policies in place that prevent their teams from becoming better at what they do. They prevent their teams from doing good work!

How can Coaching / Consulting Help?

OK, I am assuming that your company actually wants to do the right thing - they want to become more agile, and they want to do Agile right. If your company - or some important managers within - do not want to do the right thing, you have a completely different problem.

So, if your company wants to do the right thing, but you have one (or both) of the problems mentioned above, you need help.

First, you need management coaching. You need someone who can show your managers how they are mis-using the agile tools as tools for micro management. Your managers probably have valid concerns and good reasons for acting the way they do. So they need someone with whom they can discuss their concerns. Someone who can show them that other ways of working can be effective too - even more effective, sometimes.

Then your team has to start improving their technical skills. They must learn to work togehter more effectivly, produce higher quality code, create evolutionary architectures, avoid defects instead of trying to find them after the fact, ... And a technical coach can help them with that.

Yes, you can probably also do both on your own - Managers do not necessarily need a coach to change. And developers, testers and everyone else on the team do not necessarily need a coach to become better at what they are doing.

But hiring external coaches to help you can have several benefits (and internal coaches too, to a lesser extent):

  • They create a time and place where the learning can happen. When the external coach is present, people will find the time to learn/discuss/work with her. They would not dare to waste the money the company pays for a coach by not working with her.
  • People actually listen to them. For reasons I don't fully understand, some people listen when a highly paid consultant tells them something, but won't listen when their employees tell them exactly the same thing (But if you are in this situation, you probably have a problem you should try to solve).
  • They bring in a fresh perspective. Existing teams often develop a certain blindness for problems they think they cannot solve. And they also suffer "inattentional blindness" - they are just too busy with other things to see certain problems. An external coach can remind them of all the things they don't see (or do not see anymore).

Sometimes you will find a coach who can help you with both - Management coaching and technical coaching. But most coaches will either be better at agile coaching or at technical coaching.

This is the reason why I, together with some other people, started the discussion about agile coaching and technical coaching. We want to bring agile coaches and technical coaches together, so that we can provide more value togehter. And we want to raise awareness that agile coaches, together with technical coaches, can help you solve some of your problems.

What Now?

Ask yourself, honestly, "Do we have those problems? To what extent do we have them?" And if yes, talk to people. Discuss possible solutions. Think about very small, safe to fail experiments that might improve your situation.

If you think an external coach or trainer could help you with some of your problems, tell your colleagues and managers. But do not assume that they will be excited quickly. It might take a long time until you can convince everyone involved that getting external help might be a good idea.

Do you want to join our discussion about raising awareness for the need of agile coaching together with technical coaching? Do you have any questions about this discussion? Please drop me an email, I would love to hear your input and answer your questions...

The Missing Link in Agile Coaching

I was fascinated with agile software development from the moment I first heard about it in an eXtreme Programming course at University. And when I was a developer in a Scrum team for the first time, I decided that I have to become involved in this “agile” thing / community.

Because of that moment almost 10 years ago, I started to learn more about Scrum, Kanban, Software Craftsmanship and other “agile” stuff. Because of that moment, I write on this blog, give conference talks and try to help people become better at developing software.

Agile Coaching - Not for me?

And then it finally paid off: A company asked me to be a Scrum Master / Agile Coach for two of their development teams.

I was there for just about 1.5 years, and I learned so much! I was not only their Scrum master, I was also responsible for coordinating with external departments (mainly Testing and Operations).

We transformed the approach of the team from Scrum towards Kanban. I worked with their product owners, business analysts and developers to introduce specification by example / agile acceptance testing. And some more…

It was a great experience, but after some time I started to miss writing software. I started to miss discussing about software architecture and creating software designs together with my teams. I realized that doing “only” agile coaching was not the right thing for me…

Player Coach?

The next few clients hired me to do different things for them. They hired me as a software architect, as a developer or as a specialist to introduce a new technology.

And while I really enjoy those things, when I come to a new client, I also recognize things that could be improved - Things about how people work together, how the client handles requirements and defects, and so on. “Process” stuff.

So, with those clients, I tried to help them improve their process while having a different role on the team - By discussing with team members and managers, and by telling them stories about what we did in other teams I worked with.

Sometimes this worked better - some people would listen to me - and sometimes it did not work so well. But I did not think about it too hard.

Agile Coaching and Technical Coaching

I did not think about it too hard until Adrian Bolboaca tweeted:

And I thought: Yes, that’s it. We have to convince companies they have to do both: Getting better at working together (e.g. by getting agile coaching) and getting better at what they are actually doing (e.g. by getting training or technical coaching).

That tweet also started a series of events, leading to, among other things, this blog post. Gitte Klitgaard tweeted:

And then Gitte, Markus Tacker and I had a video call where we discussed what we could do to improve the situation. Basically, we want to work together as agile coaches and technical coaches to provide a better experience for clients.

Going Forward

First, we want to raise awareness for the topic. We want to start planting some seeds, so maybe more companies will realize that they can benefit from agile coaching and from technical coaching.

  • Each of us will write a blog post, you are currently reading my take on the situation.
  • We will speak at conferences about this topic. I will give a talk called “Technical Excellence” at several conferences in autumn. And we want to create conference talks together.
  • We will talk about this topic to other people in person.

We think that a major difficulty here is that many companies - or, actually, people in those companies - think they do not need any outside help.

  • The current way we do things works, so why change anything? Well, the current way of doing things is often merely sufficient. And it might stop being sufficient faster than you think...
  • Developers and testers love to try / learn things on their own. Some love to tinker and learn, and this can work very well. But it often is less efficient than getting some help. And often teams do not see certain improvement opportunities because they have developed some blind spots over the years.
  • Management sometimes does not want to admit that something needs improving, because the way things are now happened on their watch. They think admitting they need coaching means admitting that they did something wrong. So they prefer continuing the old way, without getting outside help.

We also want to expand our network, so we can bring people looking for coaches and people who provide coaching together.

And we need your input here. What do you think about agile coaching and technical coaching? Where could you need some help? Where could you help others? Let’s have a short video call or an email conversation: Contact me!

Stop the Daily Standup Meeting

So, your Scrum Master (or product owner or some line manager) is out of office. She cancels the daily standup meeting, a.k.a "Daily Scrum". Did that ever happen to you? Does it happen all the time? If yes, stop doing daily standup meetings. Yes, completely stop them.

You are not doing the daily standup meeting for the team. When it gets cancelled because a certain person is not there, you are doing the meeting for that person. There might be several reasons why this is happening:

  • Everyone is working on their own tasks / user stories, with very little need for coordination outside of the regular sprint meetings. This can cause a host of other problems, but you probably do not need a daily standup meeting.
  • People are not even working on the same project. Everyone on the "team" is on a different project, e.g. people in an agency working for different customers. So your team is not even a team!
  • Your team is working closely together all the time. They are co-located, and they always work together on a single user story. Everyone already knows what everybody else is doing, so you do not need a daily meeting where everyone is stating the obvious.

One could argue that in the first two situations, the "team" is not a real team. But that would miss the point: In all those situations, the daily scrum does not provide any value to the team. It is maily used for reporting the progress to a manager (or the product owner or the Scrum Master).

And for reporting the progress of a project (or multiple projects), the daily standup meeting is not very effective. You need to find a better way to track this progress.

Or, if said manager also already knows what's going on, it is a hollow ritual that is done for no good reason. A tradition that has surpassed its usefulness. Or, if it never was useful, a cargo cult.

So, why did I say "stop doing the daily standup" instead of "find a better way to do the daily standup" then?

I want you to stop because this is a very simple, low-risk experiment you can do. You need no preparation at all, and you will get feedback immediately. And you can roll back the experiment in less than a day, if it is not working for you.

Just stop the doing the daily standup, and make everyone record what they are missing. After two or four weeks, you'll have a very good idea which and how much value the daily standup brought to the team. Just make sure everyone is actually recording what they are missing!

By stopping completely, you get a much better idea why you need the daily standup than by trying to change it in baby steps. That's why I told you to completely stop it: To get more data, faster.

Then, when you know what you are missing, you can design the next experiment around the question: How can we get the main value back, in a simpler, more effective, less boring way? The answer to that question will be different for every team. Maybe it even means bringing the daily standup back like it was. But some teams will find a solution that works better - for them.

You may also want to read Your Daily Scrum is Killing Your Team by J.B. Rainsberger. He also advocates stopping the daily standup when you do not get any value out of it, and he also describes situations in which you'd want to bring it back.

Case Study: DevOps at Dynatrace

This article is part of the series Your Company Will Never Be Agile. It is a case study about a company that successfully introduced devops to a big team.

Some years ago, Dynatrace faced the "innovators dilemma": They had and continue to have a software product that sells very well and grows fast, but they faced the challenges to converge multiple products into one and decided to start green-field to not just converge but actually leapfrog with new approaches such as artificial intelligence to monitoring.

You always need pain for change.

Bernd Greifeneder

Their major pain was slow feedback: They had 2 major releases per year. But when they realeased the software, customers would not upgrade immediately. Some would wait for 3 months or more until they installed the new version. So it took a year or more until product management got feedback from real users. At this point, the developers were working on completely different things.

They needed faster feedback. In order to achieve this, they had to move away from software that you could install on your own server - They had to move to "Software As A Service". And they needed a different organizational structure.

So they created a second product - a new generation that almost competed with their first product, but that was positioned towards most modern cloud technologies at first! Not every management is willing to take that risk. But it worked: Now they have 2 very successful products, which are both growing.

Kill the QA Team

Even before that, when the company was much smaller, Dynatrace faced a groth pain. They started to become more agile, and they had a QA department that did mostly test automation but also some manual testing. And they could never catch up. They had to develop test tools and to test and automate tests after a feature was done. As the company grew, this became worse.

So Bernd, together with his teams, decided that a QA department cannot work when you try to move fast. They eliminated the classic approach of QA entirely and instead created a new department "TA" - "Test Automation", and this new department would only work on test automation tools and help teams automate their tests. They do not test or automate tests.

Developers have to automate all tests as they go. This means more responsibility for developers - They have to make sure that what they implemented works and that it also will work in the future. And it allows you to move faster, since the test automation is done when the feature is done.

After Dynatrace was bought, Bernd has also rolled out this approach to QA globally to other teams in the company, and it was always more successful than the old way.

This initiative was again triggered by pain: Slow feedback, which this time came from the slow test automation after a feature was complete.

Zero to Devops

The pain Dynatrace felt was slow feedback, again: When they started with the SaaS product, their operations team sometimes needed weeks to react. Everyone was unhappy. So the CTO decided: "We need to be able to implement a bugfix and deploy it to production in less than one hour". DevOps at Dynatrace was born.

At this point, the company had already undergone other major changes: They had already "killed" their QA department, and they had established a culture of continuous integration and continuous testing. Also, Dynatrace had a culture that embraced failure (fail fast, fix what's wrong, learn from it) from the start. So everyone had the feeling: "We can accept this new challenge".

To achieve this goal, they had to move operations closer to development. Developers would do all operations for the application / the cluster themselves, and automate everything. Since QA was already integrated in development, after integrating operations they had turned the organizational structure by 90 degrees: Instead of different silos for different functions, they had cross-functional groups that were responsible for a delivery from end to end.

They still had a 24h operations team (which exists in the parent company anyway), but step by step they automated all manual operations procedures. And now, their only remaining procedure is: "If the cluster cannot heal itself, call someone from development". So now they have fully automated QA, deployment, and application operations, and don't need the 24h operations team anymore.

There are extreme feedback devices all over the company: The Dynatrace ufo. These are lamps that show all kinds of pipeline problems using a color code.

After the CTO started this initiative, 150 developers in two countries were immediately affected. They faced big problems at the start, and it took them some time until they honed out all of the problems, but now they are hugely successful with it. With even more developers.

Executive Support

I think one of the reasons for the success of those initiatives is that they have support on multiple levels of the hierarchy. They often start with the CTO wanting something, and he then finds supporters in product management, team leaders and developers to achieve his goals.

Bernd told me that, when you do a new project, you have to fully rely on automation from day one. This means that you'll have very high initial costs, but they will pay for themselves at some point. But you have to get a budget for this, and he sees his task as a CTO to make sure this budget is available.

The way of the engineer: Solve all problems in a sustainable way.

Bernd Greifeneder

The DevOps initiative started with Bernd putting a stake in the ground: "We need to be able to implement a bugfix and deploy it to production in less than one hour". Most told him that this is impossible, but he found enough people from product management, development and team leaders that were willing to try. So they started with DevOps. With a team of 150 developers who were all immediately affected, half of them in another country. And even though it was "a mess" at first, after some time, it was hugely successful.

At one point, Bernd decided that the feedback from their smoke tests was too slow: "One hour for smoke tests is unacceptable. Our new target is five minutes." And then he gave the TA department a lot of time and money to change the testing technology stack in order to achieve this. Over a year later, they were at ~8 minutes.

Key Takeaways

You need long-term goals: Top management has to put a stake into the ground from time to time ("We need to be able to implement a bugfix and deploy it to production in less than one hour"). And then make sure everyone understands this goal.

You need support on all levels of the hierarchy: In order to achieve those goals, you need some first supporters on all levels of the hierarchy and in all departments.

Allow and embrace failures: Failing is OK as long as you fail fast, correct what's wrong and learn from your failure.

You have to turn your company structure by 90 degrees: To move fast, you cannot have the traditional departments (silos) for Development, QA, Product Management, Operations. You need cross-functional groups that are composed of all those functions.

Everyone needs a basic understanding of "Agile": I asked Bernd if it is an advantage when managers are engineers. He said "It is an advantage, but it is clearly not required. However every manager has to understand some basic agile principles: Why we need to move fast, why we need very fast feedback, why it makes sense to invest money in solving probelms in a sustainable way".

You really have to invest time and money: You have to solve problems in a sustainable way: You have to automate everything. You have to work on faster feedback - constantly. This takes time and money. But it will pay for it at some point.

Lead, follow, or get out of the way: Everyone has to know that this is the new way, and that it will stay like this for a long time. That they cannot wait and do nothing until the next way will be announced. That they have to follow this way when they want to stay with the company. Top management has to put a stake into the ground.

Your Company Will Never Be Agile: Budgets and Bonuses

This article is the second part of the series Your Company Will Never Be Agile. The previous part is Organizational Structure Prevents Companies from Becoming Agile.

Imagine you are a manager in a company that has annual budgets and bonus systems. A team that you are responsible for wants to work in a truly agile fashion (self-organized, re-planning every 2 weeks, working with emerging requirements, ...). Do you allow them to do it? Under which conditions do you allow them to?

Bonus Systems

People with targets and jobs dependent upon meeting them will probably meet the targets – even if they have to destroy the enterprise to do it.

--W. Edwards Deming

If your company has a "traditional" bonus system, you (the manager or the employee) will get a monetary bonus if you meet some targets (based on Key Performance Indicators - KPIs) in the current budgeting cycle. You probably also risk being fired if you do not meet your targets a few times in a row.

So you have to make sure to control every aspect that might influence the KPIs your targets depend upon. In a hierarchical organization, you basically have three possibilities: Either you tell your subordinates directly what to do, you choose their KPIs and targets in a way that they positively influence yours, or you do both (Be careful if you choose the third option: You might create a percieved conflict of interest for your subordinates - between their targets and what you have said).

With your bonus and your job depending on meeting some targets annually, you cannot allow your teams to work in a truly self-organized way. You cannot allow them to re-plan every 2 weeks. You cannot allow them to work with emerging requirements (which make the end-result unpredictable). You would lose some control over your targets. You cannot let them work in an agile way.

But what if you chose all the KPIs in your company based on continuous improvement? Then you, as a manager, would still get your bonus and keep your job, as long as all your subordinates improve continuously. Sounds good, but you probably still don't want to risk it: Your subordinates might not meet their targets, and then your bonus is gone too... So, some managers will still stick to micro-management.

Annual Budgets

When your company does annual budgeting, they need to plan ahead for at least a year - probably longer. This will be an impediment to agility. It means there are things that you cannot do in the next year, because they were not accounted for in the budget.

You will prominently notice this in the last weeks of the budgeting cycle, when managers will either:

  • Throw money out of the window because there is some budget left. They will send people to trainings, buy stuff, ...
  • Push back on everything that might cost money, because the budget is almost gone. You want to hire another member for your development team? You want to buy a team license for the profiler you need? Not now, mabe in March...

The annual budget will prevent teams, departments and the whole company from reacting quickly to changes in circumstances. It will prevent the company from achieving true business agility.

Especially in combination with targets: When the bonus or the job of some of your managers depends on the company hitting the budget, they will push very hard not to implement any changes that would cause the company to miss the target. Even if those changes are necessary for the company to remain profitable in the long run.


Most of the work that companies organize as projects today should probably not be projects at all. Most of the work should be organized as a continuous stream of value creation.

Projects create problems because they have an allocated time-frame, a budget and a plan how to achieve the project's goals (there are even more problem with projects - refer to the link above for more information). The problem here is that, within the time-frame, you are not free to change direction, otherwise the project would be challenged.

To Recap...

Annual budgets, bonus systems based on KPIs / targets and projects create incentives for some people in your organization to prevent (or delay) changes (and you cannot find the right set of KPIs to prevent this). When some people in your organization delay or prevent necessary changes, your organization is not agile.

So, in order to become truly agile, your organization has to find a way to perform well without annual budgets and bonus systems, and mostly without projects.

Case Study: Technical Excellence at Smarter Ecommerce

This article is part of the series Your Company Will Never Be Agile. It is a case study about a company that managed to stay small and agile throughout their history.

Smarter Ecommerce has been an agile company from the get-go. Peter, who was one of their first employees in 2008, was always a strong advocat for agile software development.

They started with a "process" that was mostly scrum, but after almost 10 years of continuous improvement, it now looks a bit different. In those years, they started some remarkable practices, like daily tech talks and "Freaky Fridays" (see below).

Over the years, the developers started to "radiate" agile ideas to the rest of the company, and other departments (Product management, online marketing, ...) started do adopt agile practices and to live by agile values and ideas. Now the whole company is "agile".

Smarter Ecommerce places a strong focus on technical excellence and continuous learning. But this does not mean 100% defect-free software for them. They created a culture where failure is allowed, but where they try to detect problems very quickly and in an automated way and fix the important ones right away.

A Company Based on Values

Smarter Ecommerce changes their processes on a regular basis. They run small experiments all the time (from a few weeks to a few months), the keep what currently works and change what does not work - or does not work anymore.

They do this based on their values: They want to be a great employer, they want to enable their employees to learn and grow professionally, they want their employees to take responsibility, they want to adapt their processes for their people and they want to improve continuously to stay fast and agile.

To stay agile, you always have to adapt the process for the people you actually have. As a manager, you have to develop a lot of empathy for your teams and employees.

Christian Gorbach

The key here, to me, is empathy. You need a process that works for the people you have right now. Not for the people who wrote the Scrum Guide. Not for the people you'd like to have. This process might be different for every department, or for every team. Which is totally OK - It only has to work for this single team!

Also, they actively manage process changes: They have a change process for process changes. This means that they try to make changes and run small experiments all the time. They run experiments for up to 3-4 months, then keep what works, and change what does not. They also accept that things that worked well in the past might not work anymore - They even change well-established "best practices".

To manage process changes, they create "process change tasks", assign them to individuals and track their progress.

Agile in the Whole Company

All departments at smarter ecommerce work in an agile way. They saw what was working for the developers, learned about the underlying ideas and started to adopt some of the practices, and changed others. They were actively encouraged to do this by the management of the company.

If your product management is not willing to work in an agile fashion, you will never be agile.

Peter Rietzler

But not all teams / departments use exactly the same process. They are free to experiment and change the process in a way that best fits them.

Focus on Technical Excellence

Smarter Ecommerce have a strong focus on technical excellence. They actively encourage all their developers to learn and play with new technologies (see below).

They are very eager to automate everything. "You can only scale a company with great technical automation and using great tools". They have automated quality assurance, deployments and failure detection.

But technical excellence does not mean defect-free software for them:

We have to let go of the idea of "100% defect-free software". This is expensive and slow. We have to accept failures, but have to be able to detect them quickly so we can react and learn.

Peter Rietzler

According to Peter, fixing a problem in software can be more expensive than a manual workaround or a change of a process. When you fix a problem in software - especially one that rarely happens - You have to pay for it forever, on an ongoing basis: The fix affects your architecture, your design and the maintainability of your software (more code is harder to maintain).

The key here is to detect problems as quickly as possible, in a fully automated way. And then decide quickly: Do we fix this problem right away, should we fix it later or can we ignore it?

Continuous Improvement and Learning

The company puts a lot of effort into hirign, because they want to hire people who are eager to learn. They have accepted that the cannot hire the perfect employee, but they think that they can train them.

Everyone thinks they are willing to learn, but not everybody actually is.

Christian Gorbach

They give their employees a lot of time for learning and professional development. Almost every day (but at least three times per week) they have "Tech Talks", where one of the developers talks about a new topic for 15 minutes. The developers get time to prepare the talk (this should also be done in about 15 minutes), and everybody has to give tech talks.

With this practice, they learn about many different, emerging technologies. And all the developers learn to grasp new concepts quickly, to find the important information about a new technology in a few minutes and to talk freely (almost unprepared) about it.

They also have what they call "Freaky Fridays": Every Friday, on of the developers DJ - she is responsible for playing music. All other developers get the time to work on whatever they want. They could do a refactoring they didn't have time for, read blog posts, try out a new technology - Really, whatever they want. The only thing they should not do is work on the tasks they have in the current Sprint.

Key Takeaways

Actively encourage process changes: Have a change process for process changes. Assign process improvement tasks to individuals and track their progress.

Process changes have to come from the team: Process improvements have to come from the team. This also means that every team has to be allowed to have a different process than all the other teams.

The whole company has to be agile: Don't stop at the development teams. Make sure other departments learn and understand agile values and principles, and let them design a process around that.

Give people time to learn: At Smarter Ecommerce, people not only get time to learn / experiment, they are actively encouraged to take this time. "We sometimes have to force people to stop working on the current sprint", Peter told me with a wink.

Focus on technical excellence: Automate everything. Encourage people to become better at what they do - And then give them the time and budget to achieve this.

Organizational Structure Prevents Companies from Becoming Agile

This article is the second part of the series Your Company Will Never Be Agile. The previous part is Your Company Will Never Be Agile - Intro.

In order to be good at software development, you need an organization that's optimized for software development

Samir Talwar

Samir told me that when I first told him that I wanted to do an essay / talk about "Your Company Will Never Be Agile". His Theory: Many traditional organizations have organizational structures that are optimized for something else - Banks are optimized for banking, telecommunication companies are optimized for burrying cables in the ground and routing calls, and so on.

And I think it's even worse than that: Many companies have organizational structures that are optimized for how those businesses worked several years / decades ago! They evolved over a long, long time, and now cannot change easily anymore.

If your company does not actively work on optimizing it's organizational structure for software development, you probably don't have a structure that's optimized for software development.

Impediments to Agility

How can your organizational structure affect your agility in a negative way?

Remember: We are talking about true business agility here - The ability of a business to react quickly to changed circumstances. The benchmark here is basically the lead time for a small change in direction: How long does it take from the moment you recognize a change is necessary until you actually changed direction.

Such a "small change in direction" might or might not involve software development. But if IT is involved, we need to be able to deliver the new or changed functionality as quickly as possible, or we will slow down the whole organization.

Here's how your organizational structure can impede your agility:

Wasteful activities: People do some activity - filling out forms, writing emails to different departments, ... - that does not directly provide value for your end users. This is often caused by your organizational structure - When your departments and teams are organized in a way that is not optimal for delivering value.

Waiting time: You need someting from somebody in another team, but she is overworked: Queueing. You need answers from different people in different teams / departments, so you write emails with 10 people on CC and have to wait for the discussion to finish: Coordination. You need someone to sign a document before you can start implementing, testing or deploying: Clearance.

Hand-offs: Say you have two teams or departments who have to work together to provide some functionality: Team one does the first half, team two the second half of the work. Now you have a hand-off. This does not only cause waiting, it's even worse: You lose knowledge in the handoff. You probably need written documentation to coordinate the hand-off, but then you always risk that someone will not understand it correctly. And they won't ask, because they'll simply do what the document says.

Communication bandwidth: People in the same team will communicate more and better - Even if they are not in the same room.

Departments / Silos

Many companies I have worked with so far had at least some of the functions listed below in separate departments:

Development: They implement functionality and fix defects. Maybe they do some unit testing, or even test driven development.

Release Management: Some central department that takes care what is currently released on which production and staging system and when a team / software system can do a rollout. You probably need clearance from them to release something to production or even to staging.

Operations: They deploy your application to the production systems and solve minor issues themselves (like re-starting cluster nodes). You probably need clearance from them too to do a rollout, and you also need at least one person from them during the rollout.

Quality Assurance: They test all new functionality you implement and also automate those tests.

Here's a hypothetical, but not entirely unrealistic scenario. It's a bit exaggerated, but I've seen things like that happen at past clients.

"Release Management" keeps a schedule of all releases to production. They coordinate multiple software systems, and multiple teams work on each of those systems. They also have to provide clearance (usually takes 1-2 days) for each of those releases. This means every team can only release so often - otherwise they would be completely overworked. Our team is allowed to do one release every two months.

In order to get clearance from release management, you need a changelog and a test protocol. The testing department tests most new features within 1-2 days after they are declared "finished", then they need another 1-2 days to automate the test. When they find a defect in a new feature, it goes back to development and it takes another 2-3 days until it's developed and tested again. Before each release, they also insist on 2 days of smoke testing before providing a test protocol.

Operations wants the deployment unit at least 1 day before the release, but they only accept it once there is clearance from Release Management.

So, in order to reliably do a release, we have to have a code freeze 10 days before the release: 1 for operations, 2 for clearance, 2 for smoke testing, and 5 for "test-fix defect-test again" of the last few features.

Features from earlier iterations/sprints will be delayed even longer, since they have to wait until the next release, which is up to two months away.

IT itself: Say you have four departments - A, B, C and a centralized IT department. Does it make sense to have IT as a separate department? You probably have the IT department because someone saw "synergies" in centralizing it. But often the IT department is then organized in three groups: A, B and C, where group A serves department A, B serves B and C serves C. They don't even talk much with each other. So in this case, there are no synergies, but you have complicated the communication structure between each department and the corresponding group in IT. Sometimes it might make sense to have a separate IT department, but sometimes it would be better to move IT closer to the people who need them.

Communication Paths

In many companies, the established communication channels make it hard to get the information you need quickly when you need it. And decisions take longer, because you need people from multiple teams/departments to make a decision, but you can only communicate with them through the established channels.

"Reduce the distance between problems and problem-solvers" is one of the points of "Agile Doctrine", as coined by Jason Yip. And this distance is still huge in many companies. Not necessarily the physical distance - All persons might be in the same building. But the "organizational distance" can still be big: How easy is it to communicate with all the persons you need information from?

Team to end user: I have worked with companies in the past where developers are not allowed to talk to end users. The PO does the talking. But actually, the PO talks to business analysts. But they don't talk to end users either - they talk to the boss of the end users. That's 5 rounds of Chinese Whispers between end users and developers.

Availability of key persons: Maybe you are allowed to talk to everyone you need information from, but only at certain points in time. In the past, I worked with teams where we only could talk to key stakeholders during project initiation phases - They just wouldn't have any time for us later. So the business analysts had to document everything they said in writing, and later we hoped that we understood the documentation correctly (Remember: Documentation Lies). And the key stakeholders could not change their minds without a change request.

Team to other teams: You want to do something, but in order to achieve your goal, you have to coordinate multiple people from different teams (operations, ESB, other application teams, ...). When this happens too often, it is an indication that your team structure is not aligned with your business goals.

Strategy / Goals / Budgets: Often, companies try to hide the details of the company strategy, the tactical goals and the current financial situation from their "line workers". They do so with the best of intentions: The line workers don't necessarily need to know this, so why should they spend time on learning / processing this information? The problem is that self organization cannot work without information. Decision making and problem solving takes longer, since information about the problem first has to get to a level in the hierarchy where all necessary information and power is concentrated, but it will arrive there diluted. The solution will be watered down too when it finally reaches the people who need it. And the feedback loop is very long.

Inappropriate communication channels will slow you down considerably. And if you are slow to take and implement decisions, you cannot be truly agile.

Example: The Enterprise Service Bus

If you have an Enterprise Service Bus (or ESB), this will slow down your development. You might ask: How can an architectural detail like an ESB be a problem of the organizational structure? Here's how:

So, you have different software systems and subsystems that communicate with each other over a network. To make sure that you have well-defined communication protocols in place, and also for some flexibility, you bought a piece of software that manages and channels all messages between your software systems - an ESB.

If you have an ESB, you probably also have an ESB team. The ESB does not only route messages, it also filters them, provides auditing, can translate messages, implement fallback, and so on. You need someone to implement and maintain all this functionality. Your ESB team does this - After all, it would be a mess if all teams added functionality to the ESB themselves. Also, not every team can have an ESB expert on the team, right?

This is actually an example for both "Departments / Silos" and "Inappropriate Communication Channels". Because now you have a bottleneck: The ESB team. You need them for all changes that involve any communication between any two of your subsystems. That's potentially a lot of changes - Especially if you really want to have an Evolving Architecture.

Even worse: The functionality has to be deployed to production on the ESB before it can be deployed on the subsystems. If you have scheduled deployments, like in the example above, and the ESB team deployes every two weeks, then you have to wait for your next deployment after their next deployment after all teams were done implementing the change. This adds weeks to your lead time!

End-to-End in the Value Chain

You have to "Reduce the distance between problems and problem-solvers".

You will move fastest if you have small, cross-functional teams that can work on the whole value chain - From a problem some end user has to deployed software on the production systems.

You have to turn your company structure by 90 degrees.

Bernd Greifeneder

What would have been different departments or teams in the past - Development, QA, Operations, ... - should now be different roles in cross-functional teams. Teams that can do anything that's required to deliver value to the company, along the whole value chain.

This article is part of an on-going series. Want to get notified when I write the next article in this series? Subscribe to my newsletter (you should also see a popup below) and you'll get a mail with my next article.

Your Company Will Never Be Agile - Intro

Your company wants to become agile. Most companies want that. So they hire some consultants, do a couple of re-orgs, call their line managers Scrum Masters, and so on. And then they declare success: Development is a little bit cheaper now. But many of those companies with "successful" agile transitions did not really become agile. They don't have real business agility or sustainable development or truly self-organized teams.

The ugly truth is: If your company is not already agile, it most likely will not become agile. Not within a reasonable period of time. Sure, you can do all the little rituals that Scrum mandates. You can even fully implement all the extra stuff that SAFe wants you to do. But this does not make you agile. This is not what I am talking about.

I am talking about true business agility. And many companies are struggling hard to achieve it. They are not even moving in the right direction. Even though they "sucessfully" implemented Scrum (or any other "agile" process).

Business agility is the "ability of a business system to rapidly respond to change by adapting its initial stable configuration"

Wikipedia on "Business Agility"

The problem is: It does not matter that you successfully implemented Scrum or Kanban or SAFe in your development team. It does not matter that you do continuous integration or devops or noops. If your company is not able to move fast and change direction quickly, it is not agile.

The only thing that matters is its agility: how fast can the company change when the context changes. Agility is a property of the organization, not of a team. Improving agility usually includes using agile and lean practices at team level, but it’s not the full story.

Alexandru Bolboaca and Adrian Bolboaca

And if your company is not already agile, don't wait for it to become agile in the near future. "Agile" is not a new trend. "eXtreme Programming eXplained" was published 16 years ago, as of this writing. So, if your software company is not agile yet, it is already 16 years late.

Sure, some companies can and will change. But for many companies, this will take a long time. Longer than one would reasonably wait. In this article series, I will list some reasons why many companies cannot easily become agile, even when they adopt agile practices (Spoiler: Change is hard. Most companies have processes and policies in place that prevent certain kinds of changes.). So, if you are waiting for your company to become truly agile, try to face the truth: It might never make it.

But does it matter?

Everyone is affected by Economic Darwinism. In the long run, only those will survive who meet the customer needs and demands best.

Uwe Friedrichsen

Yes, it matters. Because, as Uwe Friedrichsen explains in his conference talks about "The Next Generation of IT", economic darwinism affects every company.

Some companies already feel the heat, for others it might take years or even decades until there will be a dangerous competitor (think: regulated industries, government institutions, ...).

At some point, a company will come that meets the customers needs better and faster than all the others. You can only hope this is your company.

Here are some reasons why companies have a hard time becoming agile:

Organization not Optimized for Software Development

In order to be good at software development, you need an organization that's optimized for software development

Samir Talwar

If your company is organized in too many departments or if it has the wrong department / team structure, you will struggle to become agile. An organizational structure that is not optimized for software development will be slow because there are too many hand-offs and deadlocks/live locks in your organization.

I am talking about a demand / supply split, for example. Or separate Development / QA / Operations departments. Those are things that impede your efforts of becoming agile.

Read More: Organizational Structure Prevents Companies from Becoming Agile

Bonus Systems / Annual Budgets

Annual Budgets are, by definition, not very agile. The same goes for fixed budgets for big projects. You lay out the basics of your companies finances for the next year - or longer. Changing direction quickly is not possible, especially late when the budget runs low. You'll often hear things like "We don't have money for that" - Even for necessary changes.

Bonus Systems are even worse - They encourage local optimization. People will work against each other to rescue their bonus. They won't do anything that could endanger their bonus - Even if this endangers the future of their company.

Read More: Your Company Will Never Be Agile: Budgets and Bonuses

Middle Management is a Problem

Top managers of most companies want their company to be able to react quickly. People at the bottom of the hierarchy - Developers, testers, ... - often want to be agile. But many efforts are killed by middle management.

Don't get me wrong - they don't want to be the problem. They are probably not doing this on purpose. But they are concerned about their jobs and their bonuses. Especially in companies that do not have a great failure culture.

Coming soon: Why you need a great failure culture and the right middle management to become agile.

The Company is Killing Motivation

Many companies have policies and procedures in place that completely kill the motivation of all people involved - Managers, developers, testers, operations engineers, ... . People are not allowed or able to do good work because policies, beaurocracy or a lack of funding prevents them. So they don't try anymore - They become demotivated. It is extremely hard to change those policies, because they have been in place for very long.

Sometimes nobody even sees those problems anymore - People develop blind spots. When there is a problem you cannot solve for a long time, you don't see it anymore. But motivation suffers. And without motivation, change is not possible.

Coming soon: You need motivated people that work towards a shared goal to implement change.

It's Harder for Big / Old Oranizations

Why do many big / old organizations have so many management layers? Why do they have so many policies and processes in place? Why do they have all those departments, communication channels, escalation paths, and so on? Why did they become a bureaucracy?

Probably this didn't happen by accident. They became this way for a reason. Something bad happened, so they created a policy to prevent that in the future. They tried to save money by centralizing IT or Testing or Operations. And so on. For all the little details that build up the bureaucracy, there is a valid reason that can be found in the past of the company.

The problem here is that all those little details make up a system of policies and processes where it became hard for people to do their actual work. So you have to see if you can get rid of many of your processes and policies, even though they might make sense. And you have to make sure that you don't create processes and policies as a reaction one-time events - That you don't scar on the first cut.

Coming Soon: Why you need to relentlessly simplify your company, all the time.

Counter Examples / Case Studies

I also have two case studies of companies that managed to stay agile or become agile. They are here to act as counter-examples to the claim "Your company will never be agile". I interviewed their top management and tried to summarize what they are doing right - from my point of view.

Smarter Ecommerce: A small, international company that has been agile from the get-go. They are sticking to their values, but contstantly changing their process. They practice continuous learning and improvement.
Read the case study: "Technical Excellence at Smarter Ecommerce"

Daynatrace: A company that has managed to become more and more agile even as they grew to become a large company with market-leading products.
Read the case study: "DevOps at DynaTrace"

What now?

So, your organization does not get all the topics mentioned above perfectly right. Is all hope lost? Will you never be agile?

I'm afraid it's not that clear-cut. Noone gets everything perfectly right - At least no company I know. But every area where you still have room for improvement is an impediment for your effort in becoming more agile. It is an impediment to delivering more value to your customers faster. It is an impediment to earning more money sooner.

Noone gets everything perfectly right - But successful organizations are constantly searching for areas where they can improve. And then they implement those improvements in a rigorous, sustainable, safe-to-fail way. Even if it means making big changes to their organizational structure or policies.

Want to get notified when I write the next article in this series? Subscribe to my newsletter (you should also see a popup below) and you'll get a mail with my next article.

Employee Training: Who's Responsibility is it?

After my last article, Intrinsic Motivation and Technical Excellence, a reader of my newsletter told me that something I wrote is apparently diametrically opposed to something Uncle Bob wrote:

[...] when a company employs someone, the company is responsible for training the person to become the employee they want!

Intrinsic Motivation and Technical Excellence

Whereas Uncle Bob wrote in The Clean Coder:

Your career is your responsibility. It is not your employer’s responsibility to make sure you are marketable. It is not your employer’s responsibility to train you, or to send you to conferences, or to buy you books. These things are your responsibility. Woe to the software developer who entrusts his career to his employer.

Robert C. Martin, The Clean Coder

Are we disagreeing here? I don't think so. And if we are, then only slightly.

Your Career is Your Responsibility

Period. You cannot expect your employer to do anything only you will profit from, just because "it is the right thing". So, if you want to learn technology X because it will look good on your resumee, don't expect your employer to give you the time to do so. Your employment is basically an exchange: Money for work done. Anything more (from both sides) is icing on the cake.

Your career is your responsibility. If your employer does not give you the time and resources to learn and become better, you should be prepared to work on them on your own.

And you should think hard about whether this is really the right employer for you. As an employee, your employer is probably your single source of money. So it's also your single source of failure for your career. When they stop giving your money for whatever reason, you need to be prepared to find someone else who gives you money. So if you get the feeling that your employer is steering you towards an uncertain future, think about your options early. Don't wait until it is too late.

You cannot expect your employer to do anything only you will profit from. But your employer should do things they will profit from:

Motivated People are Your Employer's Responsibility

Employment is basically an exchange: Money for work done. Actually most employment contracts are even: Money for time at the desk (but if the employee does not do any work, they will not stay with the company for very long). It is not your employee's responsibility to love their job. Companies cannot just complain that their employees are not intrinsically motivated and do nothing else - They have to create an environment where intrinsic motivation can grow.

You, as a company, are not responsible for making your employees more marketable. But, on the other hand, your employees are not responsible for becoming exactly the employees you need. They are not obliged to learn the skills you need in their spare time. When you hired them, this became your responsiblity.

If you, as a company, do not allow your people to refactor, if you do not allow them to learn or go to conferences, if you do not allow them to grow professionally, some of them will start thinking about whether you are really the right employer for them. This is absolutely reasonable because you are their career's single point of failure.

If you want to compete in this global market, if you want to stay relevant in the future, you need innovative, creative, motivated employees. You need people who love their job. This means that you have to give them jobs they can love. You need to give them freedom and time to grow professionally. Not because it's the right thing to do, but because your profit depends on it.

To Recap...

It is not the employer's responsiblity to take care of your career. It is not the employee's responsibility to become exactly the person the employer needs. But when the employee does not show any interest, they risk being fired. And when the employer does not allow their employees to grow professionall, people will start thinking about leaving.

When people start to think about leaving because they see the company as an impediment to their career, the best will leave first. Some years ago a friend of mine was leaving his company. He told me "This is a company where only people with mortgages stay. Everyone else leaves after a year". Believe me, you don't want to be a company like that.

When employment is only an exchange of money for work, both sides suffer. When you allow people to do good work - and to learn to do good work - they will be more motivated. And because they are more motivated, you, as a company, will profit.

Intrinsic Motivation and Technical Excellence

In my last article, I wrote about how managers are responsible for creating an environment where intrinsic motivation can happen. Here is one thing that I'm sure would help a lot of companies / teams:

Allow your people to achieve technical excellence!

"Wait, we allow that. We event want our people to be excellent!", I hear you say. All companies and managers want excellent people. But many are not willing to invest anything - money, time, process changes, ... - in enabling their people to achieve technical excellence. And some companies are even hindering their people to do excellent work.

You know, most (if not all) software developers / testers / managers I know actually want to do good work. They want to create excellent software. They want to learn how to do that. They were intrinsically motivated to create better software, but their company has killed this intrinsic motivation.

For many of them, developing software is a day job. They do not want to spend tens of hours of their free time per week on side projects or reading books or doing code katas. They do not want to learn and become better in their spare time.

Which is totally OK! Because, when a company employs someone, the company is responsible for training the person to become the employee they want!

So, if you want technical excellence, train your people and create an environment where people can learn. On the job. Create an environment where technical excellence is rewarded.

Are your team members allowed to read blog post, magazines and books during their work time? What is your training budget? What is your conference budget? Do you encourage your people to speak at conferences or user group meetings? Do you host or sponsor user group meetings?

Encourage experimenting and learning, even if you know that this means people will make mistakes. Allow team members to try new things and to work on stuff that is not on the Scrum board. Allow your developers to refactor (it saddens me how often I have already heard "we don't have time for refactoring").

And talk to your people. Tell them that you want technical excellence and that you need their help to change the company / department / team in that direction. Ask them what they need, and then listen to them.

Do those things one step at a time. See what works and do more of it. Stop what doesn't. Over time, the intrinsic motivation in your people to be good at what they do will come back.

Do you have any questions? Do you have any stories about companies that do this exceptionally well (or are quite bad at it)? Please tell me!


My name is David Tanzer and I have been working as an independent software consultant since 2006. I help my clients to develop software right and to develop the right software by providing training, coaching and consultanting for teams and individuals.

Learn more...

Subscribe to Developing Software Together RSS