Why not use Waterfall?

Trying not to repeat things that can be read by Googling “Why not use waterfall?”, I pose myself the question: “Why not use waterfall?”…

Waterfall is a development process, where requirements are gathered at the start, and software is delivered at the end.  However, there are modified waterfall processes, where software is delivered in iterations.  So why not use a modified waterfall process? And why, you may ask, do (some) people say you should use agile?

You can do everything in waterfall that you can do in agile. Except you cannot change requirements. That’s the only thing you cannot do. Requirements are fixed. Everything, frequent releases, simple design, dtsttcpw, prioritizing work. Using a (modified) waterfall model, you can do all of those things. So why not do it? The reason, as already stated, is that you cannot change the requirements. The investment in the requirements gathering process has already been made. An initial design has already been made. Changing requirements at the implementation stage costs too much money. And so agile isn’t about pair programming. And it’s not about stand up meetings. It’s about feedback. And it’s about responding to feedback. Being agile means being able to change direction quickly, it means accepting that requirements are things that matter right now, not what mattered 6 months ago.

In an agile process, a number of practices need to be maintained so that agility is maintained.

1. Agility is needed in the design, so that a change doesn’t require lots of rework (simple design)
2. Agility is needed in the testing, so that a change doesn’t require a lot of manual re-testing (automated testing)
3. Agility is needed in the code, so that a change doesn’t require developers to down tools (Continuous integration)

If you don’t have a simple, easy to follow design, and if you don’t have automated tests, and you don’t have code continuously integrated, and you have accumulated technical debt to a level where you can no longer change requirements without weeks of rework – how agile are you really?

And that is the question that should be asked when you review the process on your project:  “How agile are we?”.  The answer will change as the project progresses…

I’ll admit this post doesn’t really go anywhere, but it’s always worth re-emphasizing that agility is about the ability to change requirements, and nothing else.

 

 

 

Roll your own process

There are many software development methodologies out there – some are beaurocratic, with a big design up front, some are agile, with lots of practices to follow, and some are “lean”, where you’re given guidelines on how to optimise the flow of new requirements into features. Or whatever.

Today, i’ll be looking at how to create your very own agile process, based on lean principles – imagine that! Your very own process! You then have to write your own book on it and defend it to the hills (this is left as an exercise to the reader).

Choose your process

You need to pick 1 (yep, just 1) practice from under each of the bullet points.  If a practice is repeated you should pick a different one, unless you want to be ultra-lean.

1. Eliminate Waste
   • Test driven development
   • Simple design
   • Measure velocity
   • Reduce task switching
   • Make value flow
   • JIT development
2. Create Knowledge
   • Pair programming
   • Perform retrospectives
   • Do spike solutions to difficult problems
   • Collective ownership
3. Build Quality In
   • Pair programming
   • Test driven development
   • Peer review
   • Coding standards
4. Defer Commitment
   • Incremental design
   • Short iterations
   • Refactor
5. Optimize the Whole
   • Continuous integration
   • Refactor
   • Automate everything! (Tests, build process, release process…)
6. Deliver Fast
   • Continuous integration
   • Optimize workflow (eg. limit 3 items in the “ToDo” stage)
   • Short iterations
   • No BDUF
   • JIT development
7. Respect People
   • Daily Scrum
   • Collective ownership
   • Sustainable pace

 

You should now have 7 practices. If you have fewer (eg. you’ve chosen pair programming for “create knowledge” and “build quality in”), then you’ll need to defend your process more by saying your developers are smart.

If you think your developers aren’t that smart, or you don’t trust that your set of practices will reinforce each other, go and select a couple more practices to be on the safe side.

Now argue

The most important thing is, of course, how well you argue that your new process is better than all the others. I’ll give you the process i have selected, and show you how to argue about it:

1. Test driven development
2. Pair programming
3. Peer review
4. Short iterations
5. Continuous integration
6. JIT development
7. Collective ownership

An argument may now ensue:

You: Your process doesn’t involve the customer enough – how do you make sure the correct software is written?

Me: Obviously “collective ownership” extends to the customer – they need to take equal responsibility for the development. We also have short iterations which means, of course, we have regular release to the customer, so they can be sure we are moving in the right direction. And because of our other practices, the code that’s released will always be production quality!

You: Well, that won’t work for us because…

Me: Either (a) You’re not doing it right! or (b) Obviously you need to change it slightly for your circumstances…

 

So the next time you find your current process isn’t working for you, revisit this blog post and create your own! You can swap-in and swap out whatever practices you want under 1 heading, or just do them all and say you’re doing XP!

Whilst this is tongue in cheek, the question you should really be asking yourself  is: does the process you are currently following cover the above 7 process headings. If not, is that something you can live without, or are you missing something? And you really should be doing at least those 7 things…

 

 

Beware of Resharper’s var hint

I don’t personally use Resharper, but a few developers i work with love it, so i thought i’d give it a go.

I won’t bother reviewing it, as that’s something that’s been done everywhere already, but i’d like to highlight 1 particular danger of using such tools: these tools may suggest you do things which are dangerous, and are bad programming practice.

I am posting specifically about Resharper’s hint to replace all variable declarations with the var keyword.

So i looked on the internet, and saw that other people have similar concerns about using the var keyword. I don’t share all those concerns – i think it’s worth using when a variable is created on the same line it is declared. I think it’s very dangerous to use it when capturing the return value from a method call.

This is okay:
var myList = new List<int>();

This is not okay:
var myList = peopleDataAccess.GetPeopleIDs();

If you are not familiar with C#, or especially if you are not familiar with the new features being added to C#, it’s tempting to take Resharper’s suggestions on face value.

However, in the second example above, myList is now depending on the return value of a particular method. If that method is refactored such that the return type changes, the line above will still work fine, but using the variable elsewhere in the function will probably cause a compile time error. If you were explicit and said what you expect the method to return, the compiler would point to the declaration of the variable being wrong, not the use of it.

This is a big problem if you have many developers working on a project – if you change the return type, and update all the references, other developers may still get problems when they update to the latest version on their branch and get strange “method doesn’t exist” errors at compile time. If they were to instead get an error on the variable declaration line, they can see GetPeopleIDs now returns a PeopleIDCollection object (for example), rather than a list of integers, and fix the code much more quickly.

Why make developers hunt around for the cause of the problem, when the compiler can point to the cause directly?

So the moral, if there is one, is probably: Be wary of the advice that tools are giving to developers.

Whatever happened to programming?

SCRUM is a management process – it will not improve code quality.

Writing the correct software vs. Writing the software correctly

SCRUM only helps with writing the correct software – it ensures the requirements are current, and

the project owner is kept in the loop.  It also helps the project manager to plan, which means

they can create nice diagrams. And project managers love diagrams.

How does SCRUM help with the second thing?

To write software correctly, (that is to write software which is of a high quality and is easily maintainable), a methodology needs to recommend at least:

1. Write tests (for bonus marks, write automated tests)

2. Peer review code

Having tests improves quality because…

1. It’s pretty obvious isn’t it? Do i need to write a list?… okay:

2. Unit tests (for example, tests written using NUNit) prevent new code breaking existing code

3. Unit tests make refactoring safer, which should encourage refactoring, and refactoring creates a cleaner design over time, which helps with many things i won’t go in to here

4. Unit tests, when written to highlight a bug, can prevent regressions of the same bug

5. Functional tests (when automated – tests written in Selenium, for example) mean bugs only have to be found once – this can free up tester to test other parts of the system – they don’t need to retest

6. Functional tests can prove that the code thats been written matches the requirements – in fact, all tests should be traceable to a requirement. How does this help quality? Well, if all the tests map to specific requirements, then it follows that all requirements map to tests. By knowing what the tests are that need to pass for a requirement to be considered delivered, the developer then knows, more clearly, what’s expected. And also when to stop…

There’s so many more reasons that testing is important, of course, but lets look at the second thing: peer review.

Peer reviewed code will be of better quality than non peer reviewed code because…

1. You’ll have to peer review against something (eg. coding standards) – programmers have a clearer idea of what’s expected

2. Programmers will either be proud or embarassed of their code – over time, training and guidance should make all programmers proud of their code

3. Peer review spreads knowledge of the codebase to the other developers

4. Peer review can find defects or poor practice, and can promote good practice

In fact, there’s lots of way peer review helps with quality. Pair programming is the most intimate form of peer review, but there’s always room for things to be checked.

Testing will not find things that peer reviewed code will find – for example, a previous post talked about technical debt.

Testing and peer review are essential in writing quality code. If you are working in an agile software development methodology, and you have not specified how you will (a) test, and (b) review, then your code will not be of good quality.  (Okay, maybe your code will be, but what about the new boy whos just started?)

If you search for “agile” on Google, you’ll find loads of websites telling you about the improvements to the software lifecycle that can be gained by adopting an agile methodology. It seems like everyone’s becoming a Certified Scrum Master, or wants to sell some mentoring or training on how to “adopt agile” in your organisation.

I have a few thoughts on this:

1. Agile methodologies are probably the best way to produce software that a customer wants
2. Most agile methodologies are management processes – they do not specify how code should be written

I must say that some agile methodologies do not fail on point 2, for example XP specifies how code should be written, which is good in my opinion.  Other agile methodologies – Scrum, for example – are very much management processes. They introduce nothing to help improve code quality, they focus entirely on requirements elicitation and workflow.

Specific practices need to be brought into a Scrum team to ensure code is written to a high standard – my worry for agile is that these “Certified Scrum Masters” are focussing too much on the process, and missing the 2 basic principles in software engineering:

1. Write the correct software
2. Write the software correctly

Scrum only helps with writing the correct software – it ensures the requirements are current, and the project owner is kept in the loop.  It also helps the project manager to plan, which means they can create nice diagrams. And project managers love diagrams.

How does Scrum help with the second thing? How does Scrum ensure that the software is as beautifully crafted as it can be? It doesn’t… it’s up to the team that adopts Scrum to ensure the programmers have some controls in place to handle this.

So, if you are adopting an agile methodology, and also want to write software correctly, (that is to write software which is of a high quality and is easily maintainable), make sure your methodology recommends that your programmers (at least):

1. Write tests (for bonus marks, write automated tests)
2. Peer review code

Write Tests

Having tests improves quality because…

1. It’s pretty obvious isn’t it? Do i need to write a list?… okay:
2. Unit tests (for example, tests written using NUNit) prevent new code breaking existing code
3. Unit tests make refactoring safer, which should encourage refactoring, and refactoring creates a cleaner design over time, which helps with many things i won’t go in to here
4. Unit tests, when written to highlight a bug, can prevent regressions of the same bug
5. Functional tests (when automated – tests written in Selenium, for example) mean bugs only have to be found once – this can free up tester to test other parts of the system – they don’t need to retest
6. Functional tests can prove that the code thats been written matches the requirements – in fact, all tests should be traceable to a requirement. How does this help quality? Well, if all the tests map to specific requirements, then it follows that all requirements map to tests. By knowing what the tests are that need to pass for a requirement to be considered delivered, the developer then knows, more clearly, what’s expected. And also when to stop…

There’s so many more reasons that testing is important, of course, but lets look at the second thing: peer review.

Peer Review

Peer reviewed code will be of better quality than non peer reviewed code because…

1. You’ll have to peer review against something (eg. coding standards) – programmers have a clearer idea of what’s expected
2. Programmers will either be proud or embarrassed of their code – over time, training and guidance should make all programmers proud of their code
3. Peer review spreads knowledge of the codebase to the other developers
4. Peer review can find defects or poor practice, and can promote good practice

In fact, there’s lots of way peer review helps with quality. Pair programming is the most intimate form of peer review, but there’s always room for things to be checked.

Testing will not find things that peer reviewed code will find – for example, I posted previously about technical debt.

So what to do?

Testing and peer review are essential in writing quality code. If you are working in an agile software development methodology, and you have not specified how you will (a) test, and (b) review, then your code will not be of good quality.  (Okay, maybe your code will be, but what about the new boy who has just started?)

I focussed on 2 areas which can improve code quality – there are other things that can be done too of course.

Agile is very important in ensuring that the software which is delivered is relevant to the needs of the customer – just be sure that the methodology you follow has the same care over the codebase as it does over the process.

Technical Debt

I was recently at a talk given by Gary Short (DevExpress, http://garyshortblog.wordpress.com/). The talk was not quite what i expected from the synopsis.  For those who don’t know, DevExpress write a refactoring plugin for Visual Studio – i’d assumed the talk would focus on technical debt within code.  That is, new features take longer to add to software due to design decisions made earlier on.  Refactoring is one of the things that is generally done before new code is added, as it might not be easy (or indeed possible) to add the new feature without changing how things currently work.

So, given that Gary works for a company that writes refactoring software, it seemed natural that he’d talk about software anti-patterns, and how to refactor code the remove them.  Instead, he focussed on technical debt outside of coding.  I’m still not convinced that such a thing exists (or if it does, it goes by another name).  The argument was this: Adding a new feature takes time.  Time = money.  If a new feature takes longer to implement than it would take in a perfect world (eg. because of bad design, incomplete requirements, all the testers being on holiday etc.) then the difference between the two is the technical debt.  So if a feature takes a day longer to implement, there is 1 day worth of technical debt.  (The actual value in terms of £’s of the technical debt can be calculated based on hourly wage of programmer + hourly wage of tester + hourly wage of project manager etc. The way this is calculated is irrelevant to this post, but a project manager should be able to tot it up).

So, this all seems quite straightforward, what’s my problem…?  My problem is this: Gary was focussing on technical debt that sits outside of day-to-day programming.  Imagine a meeting takes 1 hour longer than it should do. As this extra hour was not budgeted for in the current iteration, there is now 1 hour of technical debt in the project. (Again, the project manager should be able to calculate the amount of money this hour has cost, based on who was at the meeting).  To me, however, this isn’t technical debt.  This is just lost time.  Technical debt lives within the project.  If you are adding a new feature to some legacy code, that legacy code will have the technical debt built in to it.  So adding a new feature might take 2 days of refactoring, plus 3 days of adding new code. So in this case there’s 2 days worth of technical debt in the existing code. This debt will not go away – it will continue to be there until something is done about it.  Gary’s example of a meeting overrunning is not technical debt in the same way.  It’s time lost, and will mean less time for developers to work on code, but it will not mean that new features will take longer to implement. The amount of time spent implementing the feature will be the same. It is possible that the time lost in the meeting will be traded for technical debt in the code (eg. there’s 1 hour less to write the code, so some corners might be cut in the design), but i think this trading of technical debt against lost time is something that should be considered as a separate entity – assuming that time is recorded accurately within the project, velocity will already cover this.

So in my opinion, time lost is not the same as technical debt – technical debt lives within the project and could lay dormant for years in legacy code until new features are added.  Time lost is only a concern of the here and now, and whether this time lost is traded for technical debt or traded for overtime is irrelevant – it is not, in itself, the same thing as technical debt.