All right, okay, let's go.
So, I want to make the case for non-blocking code reviews as opposed to the classic commonly
accepted pull request model.
Just to be clear, I don't question the pull request model in the open source world.
It has value for the open source world because it solves the trust issue.
I do question it in the corporate environment where I think it has no value at all anyway.
So, back in 2012 quite a long time ago, so it's only recently I've started speaking about
this.
I had the opportunity to start a coaching mission to upscale the engineering skills of a
rather than of a steam.
Luckily, I wasn't alone doing that.
We were two coaches and was my friend, Martin Van Aiken and myself, which makes things
more easy.
So when we arrived with this COVID situation, we didn't expect to still discover in 2012.
No version control wasn't in place.
Yeah.
So first thing we did was introduce certain control systems, and I thought like, yeah, well,
get this maybe a better bridge too far for a team that never used any version control system
while the whole idea of having a remote-reap or a local repo, when you commit it's not in the
remote repo, it was like mind-blowing.
So I suggested we'll go for a subscription.
Because it was said that while branching and subversion is difficult, complicated, I also
suggested we use no branches at all.
Everyone is going to commit straight into the main line, well, strength for a subscription.
And that went pretty well because while the second thing we introduced right from the start,
and that was the practice of continuous integration together with team commitment that any
change had to be covered by not made the best before the unit test.
Now, because the team was nervous, well, we wanted to have some mentoring in place.
And bare programming was difficult.
The team was not particularly open to the idea, although we did some pairing from a coaching
perspective, but it wasn't part of the team culture.
So the next best thing was, well, good reviews.
And back then, I was a fervent believer of good reviews, mainly influenced by books like
good complete, facts and fantasies of software engineering and writing of Joel Spolsky.
Anyone remembers Joel Spolsky?
Yay!
Well, it says something about our age, but anyway.
And because the team was nervous, we also decided every single commit had to be reviewed.
And so because there were no branches in place while we ended up doing non-blocking
code reviews, which was, well, the realization of implementing lean principles and keeping things
simple stupid.
Not really realizing that what we were doing was actually a performant way of running
code reviews.
And in all honesty, I was quite embarrassed by our way of working compared to the rest of the industry.
So we were nowhere close to what the rest of the industry was doing at that time, using branches
and code reviews, and public has the platform.
Now, there's non-blocking code reviews were based on two key ideas.
The first key idea was to establish a continuous review process that consists of regular reviews
as part of the development process.
So the team used a command board to track work.
And every ticket on the board was a feature, not a task.
And so if you want to move tickets quickly from left to right, well, it puts pressure to keep those features small.
So the feature was a unit of work on the can-man board, but the feature was not the unit of integration as it happens with feature branch.
So the unit of integration was a commit.
And the unit of release and deployment into production was a commit or a series of commits.
Because the decision to deploy into production was still a manual decision.
We practiced continuous delivery and not continuous deployed.
Now, to ensure that code reviews happened and to track the code reviews, we introduced a review column,
which indicates this feature is ready to be reviewed.
So it's good complete, ready to be reviewed.
Now, Dave Farley has seen this before.
But it never really worked out for the teams that Dave worked with,
because it was always seen as a gate.
And research is clear about process gates.
So going slow for safety is a bad idea.
Because it introduces lots of disorders and distinctions and it drives down quality.
But in other cases, it was different.
This was not a gate.
So the feature growth, commit by commit on mainline,
but reviews were happening on the whole feature on mainline.
Now, to avoid any context switching for the team,
while we decided that whenever someone finished work before starting new work,
or at the start of the day, or at the start of the afternoon,
well, they would first check the review column to see if nothing was waiting to be reviewed.
And if something was waiting to be reviewed, well, they would review it.
There was only a single rule in place.
That was, no one was allowed to review a feature they were involved in writing.
It had always to be reviewed to buy another engineer.
There were no rules in place regarding who could review the code of who.
There was no hierarchy in place.
There was true peer reviews.
And so we had seniors reviewing the code of seniors,
and reviewing the code of juniors as one would expect.
But we also had juniors reviewing the code of other juniors, and of seniors.
Now, I wasn't expecting that those juniors would provide many commands,
but I was hoping that, well, by reading the code of more experience engineers,
they would learn something.
Although I have seen quite some interesting reviews coming from juniors,
because they asked different questions.
They asked more outside the questions, the wide questions.
So the necessary triggers for improvement.
Well, if we don't practice pool requests,
or we don't practice bare programming,
well, we end up in a situation where unreviewed code is going straight
into the removed mainline.
And so we end up with having reviewed code and unreviewed code,
setting alongside each other on mainline.
And so we have this piece of code is releaseable,
at the same time.
How do we keep them apart?
And what this releaseable even mean in that situation?
And that is the second key idea behind unblocking code reviews.
Stop the code review to be definitive for the release.
So the code review doesn't define if a code base is releaseable anymore.
It is not a gate anymore.
It is not a go-no-go decision anymore.
So the team practiced true continuous integration
with train-based development,
that later evolved towards continuous delivery
with a deployment pipeline.
And every comment that went successfully
through the whole deployment pipeline,
through every stage of the deployment pipeline,
that have been totally tested,
had the potential to arrive into production.
So the gunman board was working on the level of the feature,
and reviews were working on the level of the features,
but the software grew small commits by small commit
on mainline and every commit could end in production.
So this means that unreviewed code could end up in production,
whether the feature was finished or not,
whether the feature was reviewed or not.
So we can argue,
but we are running the risk of having bad code quality in production.
And yes, that is true,
and that happened many, many times.
And that is not a problem.
Because that quality is not a bug.
Good reviews are not there to catch bugs.
Anyone who thinks that a code review will catch a bug,
well, beliefs and fair details.
It doesn't work like that.
For catching bugs, we have automated tests,
and we have exploratory testing.
So we had team commitment that any change had to be covered
by an automated test, prefer a unit test.
On top of that, we had a decent set of,
what we call automated acceptance tests,
but people usually call end-to-end tests.
Although in our case, we are not end-to-end anyway.
Then we also had the system in place that I would call peer testing.
So we didn't have test engineers in the team,
unfortunately.
So the best thing we came up was that,
well, any feature that was implemented had to be tested
by another engineer,
not involved in implementing the feature,
like performing exploratory testing.
On top of that,
we had a decent set of static code analysis
that would break the build on any rule violation.
So the risk of working like that is not that,
well, the code will be bigger or not,
but that we might deliver lower quality code into production.
But low quality code does not impact product delivery
and even less functionality.
On the other hand, low quality code does impact the speed of change.
So it will impact how quickly we can implement
new functionality and introduce new changes.
And so it will impact our time to market.
And that is the reason why we want to have high code quality.
This is not working.
So any end-to-siscoet that works,
because the test says so.
It passed all the tests.
But that could be better.
But that is somehow always true.
The code reviews will only reduce the chances of low code quality,
but it will not,
and that will mostly depend on the quality of the code review,
but it will not eliminate that code quality altogether.
Now this worked for the steam,
because while the steam decided that every single commit
had to be reviewed at some point in time.
And whenever an issue was raised,
it had to be handled with the highest priority.
Like we dropped work to fix the issue,
so that the issue was removed as quickly as possible.
Now the benefits.
Now in order to understand the benefits,
we need to understand the concept of transaction cost,
from lean manufacturing.
So transaction cost is the cost of moving a batch of work
from one stage to another stage.
It's important to understand this,
because for the higher the transaction cost is
the more inventories created in front of the next stage.
So let's say we want to order some goods online
and the transaction cost,
so the cost to deliver the goods is three euros.
Well in that case,
we will not order a single good of one euro.
No.
We are going to order a new people of goods,
so that we end up with a total cost
that is a new people of the transaction cost
to compensate the transaction cost.
So if it takes 10 minutes to create a change
and a changes waiting for a good review,
and it takes two hours
to receive a good review,
it means that the change was waiting for 92%
of the total lead time.
Now,
this is a pretty expensive process, isn't it?
So it signals us that good reviews are costly.
And so in order to compensate the transaction cost
of the good review,
well we are incentivized to ask less often for good reviews.
And so what happens when we ask less often for good reviews,
well we are going to create more changes
people asking for the good review.
And so,
it discourages us to work in small incremental steps
and it discourages us to perform refactoring.
And if we don't refactor,
we are not paying back that.
And if we are not paying back that,
we are not paying back that,
we are going to end back the speed of change.
And so introducing new functionality will take longer.
Now,
because we are waiting for someone to be available
in order to do the good review,
we are also incentivized to create more work and progress,
because while people want to feel productive,
they are not just going to wait there,
until someone is available for a good review and do nothing.
So what will they do?
Right.
They will start new work.
And there we have the start of contact stretching
and multitasking,
and we know where this goes,
and it's not pretty.
Now because of little slow,
the more work in progress we have,
while the longer the lead times become,
and so the more time it takes
to get something onto the market.
And so we end up with delay,
delay, delay feedback.
Learning will decelerate,
we become risk adverse,
innovation falls flat,
and we go back to conserve the solutions.
And this is what happens with organizations
that practice pull requests.
They go back to conservative ways of working.
So to get things out sooner,
while we need to reduce the transaction cost.
And so we need to reduce the cost of the code review.
So the benefits, second time.
So in our case,
the transaction cost was nearly zero.
It had no impact at all on the total lead time.
Comments already went through the deployment pipeline,
and when it was successful,
it could already be in production
before a code review happened.
And so there was no reason for us
to create more changes before asking for a code review.
So we could work in small incremental steps,
and refactoring happened naturally,
and so we ended up with better code quality.
Now the fact that unreviewed code
could already be tested,
and could already be in production,
before a code review even happened,
was surprisingly a significant benefit.
Because now we already received feedback from testing,
and we already received feedback from production,
from how they used the feature,
or does not use the feature,
before we started the code review.
Eventually we didn't even have to start the code review,
if the feature was not used in production.
We should have reallicated the feature.
Now because we never have to wait for someone to be available,
we can work on functionality,
and one go from start to end,
so we can focus on one single thing.
We never start new work before finishing
and ongoing work,
and so we have no multitasking anymore.
And therefore we also have way less work in progress,
and because of little flaw,
while we have a reduce lead time,
we reduce time to market.
Now sometimes it happened that during a code review,
while we discovered that this design was completely off,
like completely wrong.
These were the times where I was like,
I wish we did some pair programming to avoid the situation.
But then again,
it wasn't a big deal,
because while the test set functionality worked,
it was already in production,
delivering value to our users,
the users were already using it.
So we had all the time to redo the design,
and do a total of rewrite,
and we could even use a feedback from production
to incorporate it in the rewrite.
Something that is very difficult with pull requests.
With pull requests,
we have two options when the design is off.
Either we impose a fix,
and during the time we fix it,
while the feature is still blocked.
We still have no feedback on whether this thing
that we just implemented is the right thing.
We could even be wrong, but we don't know it.
Or, well, there is delivery pressure,
and so we get corners,
and we let it go,
and we say, we'll fix it later,
and later, never happens.
Right.
So in our case,
no pressure, no stress, no fatigue, no burnout.
Now because,
and reviewed commits,
goes straight into the remote mainline,
and eventually arrive into production,
so a lot of trust on the team.
So we are expecting the team to do the writing,
instead of thinking that the team is stupid,
and we need a process to fix for that.
No.
We were expecting the team to write automated tests,
that they were going to write the right automated test,
and that they would write high quality automated test,
and an off automated test.
So this is applying management theory,
why over management theory x.
And then again,
well, if something wrong happens in production,
that's also not the end of the world,
as long as we have a process that allows us to fix it quickly,
and in our case, we could release it in less than an hour.
Now to be honest,
well, there is a caveat with non-blocking code reviews.
It might be difficult to introduce and regulate it in industries,
especially healthcare,
where they require a gate in the form of a code review
before going to production.
Although, I can imagine ways of introducing it,
but I never worked in healthcare,
so I never tested the ideas, so I won't share it right now.
On the other hand,
there are better ways to achieve the same,
and this is bare programming and team programming.
So with bare programming,
the bare can sign over the commit,
and this is the review,
at the moment of committing,
and they can tag the change with the identified of the engineers
involved in the bare,
and this is your evidence that a code review happens.
Now,
I still think that bare programming and team programming
are a superior delivery option,
but in many organizations,
this is a cultural stretch to introduce.
It's just difficult, it's people are not open to the idea,
the whole social aspect that is involved.
So in those cases,
non-blocking reviews strategies are
significantly better than any of the alternatives,
because we have no gates in place,
because we have no gates, we have no delays.
So deliveries are never blocked,
we have shorter time to market.
Because we have no waiting times,
while people are not incentivized
to start new work before finishing on going work,
and so we have way less work in progress,
and because of little slow,
our really times are reduced,
and our times to market are reduced.
Because we are not starting new work
before finishing on going work,
while we have less context switching,
so we have no productivity curves,
no multitasking anymore.
We have way less stress and fatigue and burnout.
Also, when we find issues in good reviews,
while we have all the time to fix them,
there is no pressure to fix,
because there is no delivery pressure.
It's already delivered.
It's already in production, delivering value.
And lastly,
well, we are not incentivized to work in bigger batches,
so we can work in small incremental steps,
and we factoring happens naturally,
it's so leading to way better code quality,
which allows us to improve the speed of change,
and again reduce lead time and time to market.
Thank you.
Yeah.
Probably you see from this one,
we have an agent, agent with the code.
Can you repeat?
I don't know.
But probably you see from this.
Ha ha.
But if you change dramatically.
Yeah.
Well, I still think you need to review the code,
because you need to understand what the AI agent generated.
I have no idea that.
Well, people.
Yeah.
Otherwise, you have an asset,
where you don't understand what happened,
and this is a huge risk.
Question here on the left.
Thank you for the presentation.
As you mentioned,
there are a few million high trust for this to work,
but what are the conditions that you see this on?
What sort of teams that this is going to work with?
Because not every code is just something that's like,
or code is late and not every code is,
that's not going to do it.
Not in the nature of things,
but sometimes that's done.
Yeah.
So, good point.
So, what is,
So, I mentioned,
it requires trust and the team,
and if I understood the question right,
you are asking what all the things are needed for this to work,
and can this work in any team.
Now, we were lucky with this team.
They were really demanding,
and they saw us like,
oh, they are going to tell us of the right thing to do.
So, this is helpful.
It's a bit more difficult when you have,
like, in very senior environments,
that always worked with full request,
to just introduce that.
Because a lot of us in the minds of,
well, first we have always worked like that,
and second we have a trust that's all good,
but in the end.
I think it's a long work process.
It's a lot of massage, I think.
Thank you.
Very quick.
Did you have that question?
You make a change,
but it turns out that needs to make quite a lot of changes,
all over the code.
So, your permit isn't really very small,
and it stops becoming big.
And then most of the tests pass,
but you'll have to support 10 different platforms,
lots of hardware configuration,
lots of support libraries,
and not all the tests pass,
all the platforms are there.
Is there any way you can still move forward,
if you come guarantee that you can fix
every platform every single thing full of time.
Maybe you can still use this workflow.
Oh, yes. So, yeah.
So, repeat the question.
It's allowed for release on these platforms,
but not on these platforms,
maybe because the testing says that.
So,
We have fixed everything right now,
because I want to look forward
on the other side,
which is so amazing.
So, yeah.
So, what?
So, what if we don't have a single comment
that no people comments,
and we need to load,
do loads of changes everywhere in the code base,
but we are deploying in many platforms,
on many different platforms,
target platforms.
So, I would suspect,
yeah, I don't know, actually.
Well, the thing is,
what work?
So, this requires a deployment pipeline
that acts as a gate.
So, if tests are failing,
it just don't go to production.
Period.
And, and, and, and,
the, so, the team implements,
stop the line.
And that means they,
they drop work and fix the deployment pipeline,
because, well,
it means we are not delivering value.
Yeah.
Thank you very much.
Thank you.