To illustrate how defenses can endanger us, let’s go back 600 years to the Battle of Agincourt.

In Managing the Risks of Organizational Accidents, James Reason uses this 1415 French vs. English battle to illustrate how even today’s high-tech defenses can endanger us.

knights

Entering the battle, the English force was lightly-clad and about one fifth the size of the French force. The French had thick plate armor that could protect them from arrows and spears. During the battle, treacherous terrain caused the two French wings to bunch together in a field. Once they were within range, the English archers hit them with a flurry of yard-long, steel-tipped arrows. While many Frenchmen died immediately in that attack, those who fell off their disabled horses lay helpless, unable to get back up due to the heaviness of their own armor! The English then made quick work of them.

Unforeseen Hazards

James Reason summarized this problem as follows:

"Defenses designed to protect against one kind of hazard can render their user prey to other kinds of danger, usually not foreseen by those who created them, or even appreciated by those who use them."

In software development, a team’s automated tests give assurance that the code works as expected, protecting programmers from inadvertently introducing defects.

So how could such a safety net endanger developers and users?

It happens when automated tests lead developers to become:

  • Overconfident: programmers assume that their automated tests check everything important, they don’t manually test new or modified code and then a defects slip into production, hurting users and slowing development.

  • Inattentive: when automated tests fail for no good reason (“false positives”), programmers gradually pay less attention to these failures, a legitimate failure is not fixed, code is pushed to production and a defect slips in, hurting users and slowing development.

  • Irritated: when automated tests are too coupled to the code’s implementation logic, tests fail frequently during refactoring, programmers get irritated by all the failing tests, they start avoiding refactoring and development slows as a result of the software design becoming increasingly complex and brittle.

  • Impeded: when automated tests run slowly, programmers stop running them during development, defects slip into the code and either slow down development or hurt users in production.

  • Distracted: when automated tests run slowly, programmers become distracted during long test runs, lose focus and ultimately get less work done.

Do any of those hazards sound familiar and were any of them unforeseen?

In the best shops, people find and remove all hazards from their development defenses.

For automated tests, this means:

  • creating fast, dependable automated tests

  • manually testing when appropriate

  • fixing all false positives

  • decoupling tests from implementation logic

  • running tests frequently.

Great development defenses increase speed, lower stress and never endanger the people they are protecting.