Eine kurze Geschichte der TestpraktikenSebastian Bergmann | 10. Oktober 2014

Dieser Vortrag wuchs und wuchs, während ich ihn vorbereitete, bis er zur

Geschichte des Softwaretestens wurde, in der immer wieder die noch ältere

Geschichte der Softwareentwicklung flüchtig auftauchte.

1947

31 Jahre vormeiner Geburt

Sebastian BergmannDriven by his passion to help developers build better software.

sharing experience

1959 - 1963

Test-First Development"Project Mercury ran with very short (half-day) iterations that were time

boxed. The development team conducted a technical review of all changes,

and, interestingly, applied the Extreme Programming practice of test-first

development, planning and writing tests before each micro-increment."

— Craig Larman and Victor R. Basili

1970

The Misunderstood Waterfall Paper"He was always a proponent of iterative, incremental, evolutionary

development. His paper described the waterfall as the simplest description,

but that it would not work for all but the most straightforward projects. The

rest of his paper describes [iterative practices] within the context of the 60s/

70s government-contracting models (a serious set of constraints)."

— Walker Royce (about his father Winston Royce)

1972

The Humble Programmer"Those who want really reliable software will discover that they must find

means of avoiding the majority of bugs to start with, and as a result the

programming process will become cheaper. If you want more effective

programmers, you will discover that they should not waste their time

debugging – they should not introduce bugs to start with."

— Edsger W. Dijkstra

1987

No Silver Bullet"Nothing in the past decade has so radically changed my own practice, or its

effectiveness [as incremental development]."

— Frederick Brooks

1993 - 2000

Test-Driven Development"Style of programming in which three activities are tightly interwoven:

coding, testing (in the form of writing unit tests) and design (in the form of

refactoring)."

— Agile Alliance

Test-Driven Development» Kein Produktionscode, der nicht durch einen Test motiviert wurde.

Niedriges Risiko, unnötigen Code zu schreiben.

» Kein Produktionscode, der nicht durch einen Test abgedeckt ist.Niedriges Risiko, Defekte nicht direkt zu bemerken.

» Kein Produktionscode, der nicht testbar bzw. sauber ist.

Sauberer Code"Sauberer Code kann von anderen Entwicklern gelesen und verbessert

werden. Er verfügt über Unit- und Acceptance-Tests. Er enthält

bedeutungsvolle Namen. Er stellt zur Lösung einer Aufgabe nicht mehrere,

sondern eine Lösung zur Verfügung. Er enthält minimale Abhängigkeiten,

die ausdrücklich definiert sind, und stellt ein klares und minimales API zur

Verfügung."

— Dave Thomas

Test-Driven Development"I have often compared TDD to double-entry bookkeeping. The act of stating

every bit of logic twice, once in a test, and once in the production code, is

very similar to the accounting practice of entering every transaction twice,

once on the asset side, and once on the liability side. The running of the

tests is very similar to the creation of the balance sheet. If the balance of

assets and liabilities isn't zero, somebody made a mistake somewhere."

— Robert C. Martin

Chicago-Style TDD

London-Style TDD

2001

The Agile Manifesto"[...] to heal the divide between development and business"

— Kent Beck

The Agile Manifesto» Individuals and interactions over processes and tools

» Working software over comprehensive documentation

» Customer collaboration over contract negotiation

» Responding to change over following a plan

~ 2002

Acceptance Test Driven Development"Analogous to test-driven development, this practice consists in the use of

automated acceptance tests with the additional constraint that these tests

be written in advance of implementing the corresponding functionality."

— Agile Alliance

Acceptance Test Driven Development"We distill acceptance criteria into automatable tests expressed in a natural

language rather than a programming language. This enables us to

completely separate the articulation of expectations from any technical

details or dependencies."

— Elisabeth Hendrickson

Acceptance Test Driven Development"We write fixtures or libraries to wire the keywords in the tests to the

software under development during implementation. That’s wiring, not

translating. And doing it as part of the implementation effort, not

attempting to retrofit automation after the code is written."

— Elisabeth Hendrickson

2003

Behavior-Driven Development"I was trying to teach TDD and found I kept getting bogged down by

irrelevant conversations about testing. So I changed the words."

— Dan North

2004

Behavior-Driven Development"[I] made the connection between how I was describing the-thing-Kent-

calls-TDD and how he liked to structure conversations with his stakeholders.

That’s when the given-when-then vocabulary appeared."

— Dan North

Behavior-Driven DevelopmentGiven some initial context

When an event occurs

then ensure some outcome

2005 - 2009

Behavior-Driven Development"At this point [...] I was describing BDD as an outside-in, multiple layer,

multiple stakeholder methodology (as distinct from XP with its single on-

site customer)."

— Dan North

2009

The Corruption of Agile"Unfortunately the deemphasis of practices within the Agile movement has

only served to widen [the divide between development and business]. While

project managers have flocked into the Agile movement, developers have

fled out of it. The original movement has fractured into two movements.

The Software Craftsmanship movement has preserved the coupling between

practice and culture; whereas the Agile movement has shifted away from it."

— Robert C. Martin

Manifesto for Software Craftsmanship» Not only working software, but also well-crafted software

» Not only responding to change, but also steadily adding value

» Not only individuals and interactions, but also a community of professionals

» Not only customer collaboration, but also productive partnerships

External and Internal Quality"Running end-to-end tests tells us about the the external quality of our

system, and writing them tells us something about how well we [...]

understand the domain, but end-to-end tests don't tell us how well we've

written the code. Writing unit tests gives us a lot of feedback about the

quality of our code, and running them tell us that we haven't broken any

classes [...]"

— Steve Freeman and Nat Pryce

~ 2010

Behavior-Driven Development"Behaviour-Driven Development is a synthesis and refinement of practices

stemming from Test-Driven Development and Acceptance Test Driven

Development."

— Agile Alliance

Behavior-Driven Development

— Agile Alliance

» Apply the "Five Why's" principle to each proposed user story, so that itspurpose is clearly related to business outcomes

» Implement only those behaviors which contribute most directly to thesebusiness outcomes, so as to minimize waste

» Describe behaviors in a single notation which is directly accessible todomain experts, testers and developers, so as to improve communication

» Apply these techniques all the way down to the lowest levels of abstraction

Behavior-Driven Development"[T]he Rails community has adopted the strategy of writing hoards of

cucumber tests that drive the application through the web server. This leads

to slow and fragile tests and breaks many of the rules of TDD which tries to

keep tests from coupling to unnecessary system components like the GUI."

— Robert C. Martin

Behavior-Driven Development"If you are doing BDD, you are doing TDD, and all the rules of good test

design, and decoupling still apply. You don’t get a special license to create

highly coupled tests just because you are doing BDD."

— Robert C. Martin

TDD is dead. Long live testing."Enough. No more. My name is David, and I do not write software test-first.

I refuse to apologize for that any more, much less hide it. I'm grateful for

what TDD did to open my eyes to automated regression testing, but I've

long since moved on from the design dogma."

— David Heinemeier Hansson

TDD is dead. Long live testing."I rarely unit test in the traditional sense of the word, where all

dependencies are mocked out, and thousands of tests can close in seconds.

It just hasn't been a useful way of dealing with the testing of Rails

applications. I test active record models directly, letting them hit the

database, and through the use of fixtures. Then layered on top is currently a

set of controller tests, but I'd much rather replace those with even higher

level system tests through Capybara or similar."

— David Heinemeier Hansson

Monogamous TDD"If you trust those integration tests so much that you are willing to deploy

when they pass; and if they execute so quickly that you can continuously

and effectively refactor and clean the code, then you aren't doing any better

than me. Do it."

— Robert C. Martin

Monogamous TDD"But (and this is a big "but"), it seems to me that integration tests have very

little chance of meeting my two predicates."

— Robert C. Martin

Effective Testing

— Rich Martin

» A high-fidelity test is one which is very sensitive to defects in the codeunder test

» A resilient test is one that only fails when a breaking change is made to thecode under test

» A high-precision test tells you exactly where the defect lies

talks.thePHP.cc

sebastian@thePHP.cc

@s_bergmann

sharing experience