Agile Software Development

Preamble

development activities
Figure 1. Methods organize Activities
Methods are guidelines and rules for activity organization

Agenda

  • Introduction

  • Agile Development Tools

  • Agile Development

  • Extreme Programming

  • Test-driven Development

  • Conclusion

Introduction

Rapid Software Development

  • Rapid development and release are often the most important requirements for software

    • Businesses operate in a fast–changing requirement and it is practically impossible to produce a set of stable software requirements

    • Software has to evolve quickly to reflect changing business requirements

  • Classic plan-driven development is essential for some types of system but does not meet these business needs

  • Agile development methods emerged in the late '90s whose aim was to radically reduce the delivery time for working software systems

Agile Development Characteristics

  • Pragmatic development method

    • Rapid feedback on development status

    • Short development cycles

  • Design and implementation are interleaved

  • Incremental development

    • No heavy prediction whose errors would be found later

    • The system is developed as a sequence of versions or increments

    • Frequent releases

    • Fast development feedback

Agile Development Characteristics (Cont.)

  • Adaptation

    • Regular adjustments

    • Flexible planning

  • Extensive tool support (e.g. automated testing tools)

  • Minimal documentation: focus on tests and high quality code

  • High human implication

    • Stakeholders are involved in version specification and evaluation

Agile vs. Plan-Driven Development

plan driven development
Figure 2. Plan-driven development
agile development
Figure 3. Agile development

Agile vs. Plan-Driven Development (Cont.)

Plan-driven development

A plan-driven approach to software engineering is based around separate development stages with the outputs to be produced at each of these stages planned in advance. Not necessarily waterfall model – plan-driven, incremental development is possible Iteration occurs within activities.

Agile development

Specification, design, implementation and testing are inter-leaved and the outputs from the development process are decided through a process of negotiation during the software development process.

Agile Methods

  • Dissatisfaction with the overheads involved in software design methods of the 1980s and 1990s led to the creation of agile methods. These methods:

    • focus on code and tests rather than heavy design

    • are based on an iterative software development approach

    • are intended to deliver working software quickly and evolve this quickly to meet changing requirements

  • Agile methods aim at reducing overheads in the software process and to be able to respond quickly to changing requirements without excessive rework

Agile Manifesto

  • We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

    • Individuals and interactions over processes and tools

    • Working software over comprehensive documentation

    • Customer collaboration over contract negotiation

    • Responding to change over following a plan

  • That is, while there is value in the items on the right, we value the items on the left more.

Agile Methods Principles

PrincipleDescription

Customer involvement

Customers should be closely involved throughout the development process. Their role is provide and prioritize new system requirements and to evaluate the iterations of the system.

Incremental delivery

The software is developed in increments with the customer specifying the requirements to be included in each increment.

People not process

The skills of the development team should be recognized and exploited. Team members should be left to develop their own ways of working without prescriptive processes.

Embrace change

Expect the system requirements to change and so design the system to accommodate these changes.

Maintain simplicity

Focus on simplicity in both the software being developed and in the development process. Wherever possible, actively work to eliminate complexity from the system.

Agile Method Applicability

  • Small or medium-sized product development

    • Virtually all software products and apps are now developed using an agile approach

  • Custom system development within an organization, where there is a clear commitment from the customer to become involved in the development process and where there are few external rules and regulations that affect the software.

Agile Method Examples

  • Rapid Application Development - RAD [James Martin, 1991]

  • eXtreme Programming - XP [Kent Beck, 1999]

  • Test-Driven Development - TDD [Robert C. Martin, 2007]

  • Scrum [Ken Schwaber, 1995]

  • Lean [Poppendieck, 2003]

  • Kanban [David Anderson, 2010]

Agile Development Tools

Automation Tools

For unit tests:
  • Easy unit tests writing

  • Fast test execution

  • Re-execution facilities

Integrated Development Environment
  • Improves coding: autocompletion, method generation, code formatting

  • Quick test execution

  • Refactoring

  • Examples: NetBeans, Eclipse, IntelliJ IDEA.

Automation Tools

Build tools:
  • Automatic test execution

  • Automatic report and website generation

  • Maven, Gradle

Test evaluation
  • PIT, Cobertura

Static code analysis
  • PMD, Checkstyle, Findbugs

Continuous integration tools:
  • Sonar, Jenkins, Travis

Collaboration tools
  • Mattermost, Slack

Gradle / Maven

  • Repetitive task automation

  • Sequence of different stages of building

PMD

  • Open-source tool for static code analysis

  • Available as a plugin for most IDEs (Eclipse, Netbeans, JEdit, IDEA, etc.)

  • Available for Maven

  • Detects:

    • Dead code: unused variables, parameters, and methods

    • Unused objects

    • Empty code blocks: try / catch / finally / switch

    • Duplicate code

    • Unnecessarily complex expressions

Checkstyle

  • Open-source tool for checking code formatting and presentation

  • Support different code conventions

  • Available as a plugin for most IDEs (Eclipse, Netbeans, IDEA, etc.)

  • Available for Maven and Gradle

  • http://checkstyle.sourceforge.net

Extreme Programming (XP)

XP

XP is a lightweight development method for small (or even medium) software development teams whose needs are vague or change frequently.

Why Extreme?

Pushing common sense to the extreme:
  • Since code reviews are good, do it all the time (pair programming)

  • Since tests are good, test all time (unit testing) and include clients (acceptance testing)

common sense
Figure 4. Common Sense

Extreme Common Sense

  • Since design is good, make it everyone’s daily occupations (code refactoring).

  • Since simplicity is good, make the system have the simplest design to meet current requirements (do the simplest thing that can possibly work).

fixie bernard
Figure 5. Fixie Core Line 4130 Bernard

Extreme Common Sense (Cont.)

  • Since architecture is good, make everyone define and refine it all the time (metaphor).

  • Since integration test is good, integrate and test several times a day (continuous integration).

Leaning into the afternoons, I cast my sad nets towards your oceanic eyes.

Leaning into the afternoons
— Pablo Neruda

Extreme Common Sense (Cont.)

  • Since short iterations are good, do iterations that are really, but really very short: seconds, minutes or hours, not weeks, months or years (the planning game).

short iterations

Promises to Developers

  • Developers will work on important things every day.

  • They will not face frightening situations alone.

boo code
Figure 6. Scary Code

Promises to Developers (Cont.)

  • Developers can do everything in their power to make the system work

  • They will make decisions for which they are best qualified and will not make decisions for which they are not.

Promises to Customers and Managers

  • They will receive the most possible value out of every programming week

  • Every week they will be able to see concrete progress on the goals they care about

  • They will be able to change the direction in the middle of the project without incurring exorbitant costs

Summary of XP’s promises

  • Risk reduction

  • Improved response time to changes

  • Improved productivity throughout the development cycle

  • Bring a little fun to team-based software development.

A different development method

incremental planning
Figure 7. Incremental Planning
XP differs from other methods by:
  • its short cycles, which give early, concrete and continuous feedback.

  • its incremental planning approach.

  • its ability to flexibly plan the implementation of new functionalities and respond to changing needs.

A different development method (Cont.)

  • XP relies on both automated tests and acceptance tests for the evaluation of the evolution and for the early detection of errors

  • It also relies on oral communication, testing, and source code to describe the objectives and structure of the system

A different development method (Cont.)

  • XP is based on an evolutionary design process, which lasts as long as the system.

  • XP relies on close collaboration between regular programmers.

  • It relies on practices that work with both: the short-term instincts of programmers and the long-term interest of the project.

A Software Engineering Discipline

  • For projects carried out by small development teams (between 2 and 10 people), without strong hardware constraints and where tests can be carried out in less than half a day.

  • XP may scare or annoy some people, but its ideas are not new.

The problem: a question of risks

Typical risks

  • Project slips

  • Project cancellation

  • Systems goes sour

  • Defect rate

  • Business misunderstood

  • Business changes

  • False feature rich

  • Staff turnover

dynamite

Detailed Typical risks

Schedule slips

the day for delivery comes, and you have to tell the customer that the software won’t be ready for another six months.

Project canceled

after numerous slips, the project is canceled without ever going into production.

System goes sour

the software is successfully put into production, but after a couple of years the cost of making changes or the defect rate rises so much that the system must be replaced.

Defect rate

the software is put into production, but the defect rate is so high that it isn’t used.

Detailed Typical risks (Cont.)

Business misunderstood

the software is put into production, but it doesn’t solve the business problem that was originally posed.

Business changes

the software is put into production, but the business problem it was designed to solve was replaced six months ago by another, more pressing, business problem.

False feature rich

the software has a host of potentially interesting features, all of which were fun to program, but none of which makes the customer much money.

Staff turnover

after two years, all the good programmers on the project begin to hate the program and leave.

Reduce Schedule Slips

  • Short cycles.

  • The customer checks the progress of the work every one to four weeks.

  • A job lasts between one and three days.

  • The most important features are implemented first: in case of a delay, features that are not present will have a lower priority.

Avoid Project Cancellation

Developers ask the customer for the smallest delivery of the system which gives it an elementary meaning
  • If the project fails, it will fail quickly 😊

Prevent System to go Sour

  • A comprehensive test suite is executed and rerun after every change, to ensure quality baseline

  • The system always remains in prime condition.

Avoid Heigh Defect Rate

  • Tests are written both by programmers (method by method) and by clients (functionality by functionality).

Avoid Business Misunderstanding

  • The customer must be part of the team.

  • The project specification is continuously refined: the common know-how of the customer and the programmers is reflected in the software.

Managing Changing Requirements

  • Since the cycles are short, there is little change in requirements between the start and end of a cycle.

  • After every release, the customer can replace some unimplemented functionality with new ones.

False Feature Rich

  • Only the highest priority tasks are addressed.

Reduce Staff Turnover

  • Each programmer estimates the time needed to complete a task and receives feedback to improve his or her estimative

  • The programmer will not be frustrated by the assignment of tasks that are underestimated or even impossible

  • Encouragement of human relations within the team

    • reduced loneliness and dissatisfaction.

  • Pre-established pattern of change:

    • newcomers accept more and more responsibility and are helped by more experienced programmers

A Development Episode

An extreme day

  1. Choose a task

  2. Choose a pair

  3. Discuss the previous day’s work

  4. Write tests

  5. Implement and test

  6. Write more tests.

  7. Refactoring, design improvement and tests.

  8. Integrate and test.

Conclusion of the day

  • Pairs program together

  • Development is test-driven: test first, code later

  • A task is not finished until all the tests succeed

The pair is not restricted to the execution of tests:
  • It evolves the design of the system

  • Changes are not limited to one region: they can be made anywhere in the code.

  • The pair improves analysis, design, implementation and testing.

  • The integration and its tests are done right after the development.

Method Overview

  • Planning game

  • Short release times

  • Metaphors

  • Simple design

  • Tests

  • Refactorings

  • Pair programming

  • Collective ownership

  • Continuous integration

  • 40 hours per week

  • On-site costumer

  • Coding conventions

Planning game

Managers decide:
  • The scope

  • Priorities

  • Release composition

  • Release dates

Developers decide:
  • Estimations

  • The consequences of technical choices.

  • The process (organization).

  • The detailed planning (high-risk tasks first, flexible enough to take into account the costumer’s needs).

Short Release Times

A release must:
  • contain the customer’s most important requirements

  • be consistent: we don’t do the job by halves to reduce the time to production

  • have a lead time of one or two months rather than 6 months or a year

Metaphors

  • Each project is guided by a single metaphor: the computer should look like a desk, calculating pensions is like a spreadsheet, etc.

  • Metaphors replace traditional development architectures.

  • They bring an overall cohesion that is easy to share.

Simple Design

A good design:
  • passes all tests.

  • has no duplication.

  • declares that every intention is important to programmers.

  • contains as few classes and methods as possible.

Tests

  • Programmers write tests to trust their code.

  • Customers write tests to trust the functionality of the system.

  • The result is a reliable program that accepts changes

  • It is not necessary to write one test per method.

Code Refactoring

  • Design simplification after adding a feature.

  • The new design must pass the tests

code refactoring

Pair Programming

1 computer, 2 programmers.

The first one thinks locally:
  • What is the code needed to implement a method?

The second one thinks strategically:
  • Will the approach you’re taking work?

  • Which test cases will not work?

  • Isn’t there a more global solution to solve this specific problem?

pair programming
Figure 8. Pair Programming

Collective Ownership

collective ownership
Figure 9. Collective ownership
  • Any programmer can change any piece of code at any time.

  • Every team member is responsible for the entire system.

Continuous Integration

  • The code is integrated and tested after a few hours, a day at the most

  • Use a dedicated machine for integration

  • Integration tests must pass. If they don’t:

    • Solve the problems

    • Start development from scratch

40 hours per week

  • Or, 35 (🇫🇷)

  • You have to be fresh and passionate to work well.

  • No one can work 60 hours a week, for more than a week, and stay fresh, creative, careful and confident.

Onsite Costumer

  • To answer questions, resolve conflicts, set low-level priorities.

  • Expensive, but brings a lot to the project.

Code Conventions

  • If anyone can change the code, everyone should use the same convention.

  • They have to be adopted by the whole team.

Conclusion

Tests are very important:
  • Tests passed = confidence.

  • More confidence = more efficiency.

Extreme Programming Particularity:
  • XP is against the hypothesis: "the cost of change increases exponentially over time".

  • On the contrary, it is better to delay any decision that is not sure.

Conclusion (Cont.)

  • Simple design

  • Automated testing

  • Practice in design modification.

Test-Driven Development

Traditional development

developpement traditionnel

Test-driven development

tdd 1

Test-driven development

tdd 2

Test-driven development

tdd 3

Test-driven development

tdd 4

Approach

  1. Write test cases before the software unit

    • Test cases describe what the unit should do

    • Before writing the code, tests fail

  2. Implement the code that makes the test case pass

    • After the implementation, test cases should pass

Development Style

TDD Mantra : red/green/refactor
  1. Write tests and make them fail

  2. Write code to make tests pass

  3. Refactor

Consequences

  • Test cases specify what the unit should do, but not how.

Consequences

  • First, developers think about how to use the unit.

  • Second, they think about how to implement it.

Consequences

It is a development approach that results into units that are easy to:
  • Unit test

  • Enhance and adapt

Consequences

Developers know at any time:
  • What works as it should

  • What does no longer work as it once did

Motivations

  • Developers never have free time to write tests

  • Tests help developers to overcome complexity.

    • Reduces developer’s frustration

Plan

  • Introduction.

  • TDD and unit tests

  • Strategies.

  • Conclusion.

Development Cycle

Five steps cycle:
  1. Write a unit test

  2. Make it fail

  3. Write enough code to make it pass

  4. Check that the test passes

  5. Eliminate duplicate code, improve design

Benefits

Test cases become a support to documentation:
  • They specify the expected behavior

  • They show examples of code usage

Benefits

YAGNI (You Ain’t Gonna Need It).

  • Developers only write necessary code

    • They only code when a test fails

  • No code without test and no test without a user requirement

Benefits

  • Rapid feedback on code quality

  • Short development cycle iterations

  • Code is executed straight away

Benefits

Automatic unit tests are reproducible:
  • They can be used as non-regression tests and after refactoring.

  • They reduce fear on changes

Plan

  • Introduction.

  • TDD and unit tests

  • Strategies.

    • Development

    • Debugging

    • Maintenance

  • Conclusion.

Development Strategy

  1. Test often

    • to find errors quickly

    • to know when it is finished

  2. Move forward by small steps

    • programming and refactoring

  3. Use automation tools.

Development Strategy

tdd 5

Plan

  • Introduction.

  • TDD and unit tests

  • Strategies.

    • Development

    • Debugging

    • Maintenance

  • Conclusion.

Debugging Strategy

  • Repeat:

    1. Find the most annoying defect

    2. Write a unit test that exposes it

    3. Correct the error

    4. Test

    5. Refactor

    6. Test again

Debugging Strategy

debug

Plan

  • Introduction

  • TDD and unit tests

  • Strategies

    • Development

    • Debugging

    • Maintenance

  • Conclusion

Maintenance

Maintenance

Legacy code:

— Does someone remember who wrote this code?

— Last time someone touched this, we needed 3 weeks to make it work again.

— The documentation is available, but it does not correspond to the code anymore.

Working with legacy code

  • Developers must take small steps, like a tightrope walker

  • Without forgetting the safety network:

    • Unit tests, versioning, backups

Maintenance approach

legacy

Maintenance approach

  1. Try to understand the legacy code. And don’t.

  2. Try to write a unit test based on what they understood. The test fails.

  3. Improve the test until it passes

  4. Restart with the next unit.

Plan

  • Introduction.

  • TDD and unit tests

  • Strategies.

  • Conclusion.

Conclusion

  • Requirements drive the tests.

  • Tests drive the development of the application code.

  • No application code is written without writing a failing test first.

  • Tests are executed often.

  • Tests and code are written in elementary increments.

  • Refactoring becomes a continuous operation and is supported by a passing a suite of tests.

Benefits

  • Reduces debugging time (and bugs)

  • Short development cycles

  • Tests become part of the software product

  • Improve documentation

  • Improve code quality

  • Reduce maintenance costs

  • Increases development speed (less time is spent chasing bugs)

References

  • «Extreme programming explained: embrace change». Kent Beck. Addison-Wesley, 1999.

  • Kanban: Successful Evolutionary Change for Your Business : David Anderson ISBN 978-0984521401 (existe en français http://tinyurl.com/LivreKanban) 

  • Product Development Flow : Don Reinertsen - ISBN 978-1935401001

  • Kanban pour l’IT : Laurent Morisseau - ISBN 978-2100578672

  • Personal Kanban : Jim Benson - ISBN 978-1453802267

References (Cont.)

  • Scrumban : Corey Ladas - ISBN 978-0578002149

  • Kanban for the skeptics : Nick Oostvogels https://leanpub.com/kanbanforskeptics

  • Kanban and Scrum : Henrik Kniberg, Mattias Skarin ISBN 978-0557138326

  • The Goal : Eliyahu Goldratt (1984) ISBN 978-0884270614

  • The Lean Startup : Eric Ries ISBN 978-0307887894