The significance of the book "Extreme Programming Explained" in agile software development

Many years ago, I came across this book through a recommendation, and reading it with those eyes, I loved its approach. It synthesized all the discussions we developers had by the coffee machine back in the '90s (well, except for the fact that I was still a newbie) due to the poor business practices we often suffered as a result of an industry still maturing.

While XP could have been considered little more than an experimental essay on project management back then (the Agile Manifesto didn't even exist at the time), its influence on the Agile culture or philosophy (or whatever you want to call it) cannot be denied today. The so-called "agile" methods and principles truly caught the attention of many companies concerned with software development just a few years after this book was published, at least in our country.

"Extreme Programming Explained" is the first book on XP and perhaps the most important one you can read on the subject. Considering that it reflects its creator's perspective, it is appreciated that it manages to convey his enthusiasm for the topic. Revisiting it from time to time is also beneficial because it is one of those books where no paragraph is superfluous, and it is worth revisiting it with already assimilated concepts.

Why was it an innovative book?

What was truly innovative back then was the approach that completely changed the way projects were planned, based on the idea of team interactions over processes. XP proposes that project planning is done in two parallel phases: high-level planning and detailed planning for each iteration. The first aims to determine the functionalities that will be implemented in the immediate future and is usually done in collaboration with the client. This is commonly known as defining user stories with the client.

But the essence lies in the existence of a second planning phase, which can be considered in the background and iterative. After having user stories defined, it is necessary to create a release plan that indicates the user stories that will be implemented for each version of the software and the dates on which these versions will be released. Each release plan is a planning where developers and clients establish the ideal implementation times for user stories, the priority with which they will be implemented, and the stories that will be included in each version of the software. Only user stories that have passed the acceptance tests are put into production. These tests are defined beforehand with the client.

This broke away from what many of us had been taught in university. XP proposed dividing every project into iterations (of approximately 3 weeks in duration) and leaving aside the old phases of the waterfall life cycle, stacks of documentation (for the seasoned veterans: do you remember Metrica3?), or plans that were never fulfilled. Instead, it focused on tasks that lasted only a few days but immediately added value to the project once completed. These tasks allowed for quick detection of specification changes and enabled the team to adapt rapidly to those changes.

The problem we sometimes encountered with the classic approach (the waterfall life cycle) was that resolving a problem involved a cost increase that could be exponentially higher in certain phases compared to others. So, for every monetary unit spent on fixing something in the early stages of the project, you could end up spending 1,000 times more if it went into production without being detected. All this was coupled with an inflexible life cycle where communication between the parties involved was not fluid enough. That's what XP is all about---nothing more. It's about the need to improve communication between the development team and the rest of the business. And that's not easy, but an added challenge that helps us achieve excellence.

Is XP all that wonderful?

In my opinion, it may be for some types of projects, but I have my doubts about others. For example, a type of project where XP may not be suitable (here, I let my imagination run wild) could be a critical security system for a nuclear power plant, where cost and refactoring changes can be extremely high. Or certain "legacy" projects where their structure does not allow for the application of certain methods or rapid feedback. And of course, it should be noted that many clients demand a perfectly detailed, inflexible, and complete specification of requirements from the start, with no room for negotiation.

However, I don't want to generalize or criticize XP. On the contrary, as engineers, our job should be to assess different scenarios and decide which solution is most appropriate for each, based on technical criteria rather than trends. XP brings many benefits and a breath of fresh air compared to the classic waterfall approach. But it is also true that in recent years, with the rise of Agile, more and more development teams have tried to adopt these methodologies, sometimes without a clear understanding of the implications. What's worse is that attempts have been made to scale it to other levels of the organization without first achieving that understanding. Of course, this has left many casualties along the way. Ultimately, what lies behind methodologies like XP is the need to improve communication between the development team and the rest of the business. And that is not an easy task but an added difficulty that will help us achieve excellence.

Sometimes, we promote technologies or methods that have worked well for us, and unintentionally create dogmas. This book sometimes falls into that trap, and we shouldn't deny it. Kent Beck, with all his talent (he is a key and extremely important figure in our industry), acts like a good American (ironic) by selling you what brought him great success in companies like Chrysler. He even relies on a co-author who is a psychology expert to fuel and confirm XP's emphasis on interpersonal communication. In other words, he presents it very well, but he is also selling it to you. There's nothing wrong with that, but be careful not to finish the book believing that what it presents is the only possible truth.

Another criticism I can make of XP is that, in my opinion, it focuses too much on practices (one of its three fundamental pillars, described in more detail in the book). This may cause the methodology to become dogmatic. It is true that it presents the introduction of these practices (such as "Pair Programming") as hypotheses to decide if they are appropriate for our team. However, let's be honest: the impression you get after finishing the book is that if you are unable to carry out these practices, you are failing, or at least "falling short." In fact, it establishes two levels: essential practices that are basically indispensable, and another level that is only recommended once you have mastered the first ones. But well, the reality is that most of the practices it proposes are so consolidated today that they are hardly debatable.

At this point, I don't want you to think that I wrote this to criticize XP. I'm just pointing out what I consider to be some of its weaknesses, which, as you can see, are not that serious. XP has made undeniable contributions that still endure today: its focus on interpersonal relationships, concern for team learning, continuous feedback between the client and the development project team, adaptability to change, developer-centric approach, promotion of best practices... This book was absolutely revolutionary, considering it came out two years before the famous Agile Manifesto. We're talking about a methodology that started here and is still relevant 20 years later.

Conclusion

If you want to delve into the world of Agile, it is essential to learn about XP. Although it has been displaced by other trendy methodologies today, don't rule out the possibility of it experiencing another golden age because it has aged quite well, and its influence has been decisive.

If, after reading this book, you want to delve deeper, I can recommend other complementary books, but I believe you should start with none other than "Extreme Programming Explained."

Books for further reading:

• Kent Beck and Martin Fowler. Planning Extreme Programming. Reading, Addison Wesley, 2000.
• Agile Modeling: Effective Practices for Extreme Programming and the Unified Process. New York, John Wiley & Sons Inc. 2002.
• Fowler, M., Beck, K., Gilicze, B., Nagy, D., & Vlaskovitz, D. (1999). "Refactoring improving the design of existing code". Addison-Wesley.
• Adaptive Software Development: A Collaborative Approach to Managing Complex Systems. James A. Highsmith. Addison-Wesley.
• "Agile Manifesto"

Some interesting papers on the topic:

• Why Software Fails. Charette, R. N. (2005, 09).
• Get Ready for Agile Methods, with Care. Barry Boehm. University of Southern California.
• Challenges for Stakeholders in Adopting XP. Proc. 3rd International Conference on eXtreme Programming and Agile Processes in Software Engineering-XP in 2002.