Question
Admittedly, my own experience in cathedral vs bazaar software development is limited. The projects that would most contribute to my own opinions would be my experience working in group projects (bazaar) as well as writing NLP code for a mental health startup.
Building a Dots and Boxes Go server hosted on AWS was a great demonstration of the effectiveness of the bazaar model. It broke one of the main requirements for a bazaar: begin with a functioning code base. Sabin implemented a Go server on his own, so in a sense he provided the base. We each took ownership of a part and pushed code that was buggy but worked. Often, the solution to a bug was spotted by someone other than the person who wrote the code in the first place. Furthermore, having direct access to the codebase allowed us to reference specific lines of code and methods to call when discussing code function. Hell, I didn’t even need to completely understand how all of the server code worked. Sabin just pointed me to a method I should use to output a move for the AI, and I built the AI algorithm in there. Quick and buggy worked great to pump out code and understand problems.
My experience building an NLP model did not go as well. I had relatively little knowledge about NLP software, I was working on the code alone, and the project also suffered from a blank slate code base. I wanted to make the highest performing emotional valence detecting system I could, so I tried implementing BERT, which was a recently developed system at the time. The complexity was too high for me to understand, and the detector accuracy lagged. Obviously the project was somewhat set up for poor results, but others working on that project could have provided basic insight like 1) start simple and build to complex, or 2) start with an existing code base. Admittedly this project isn’t an amazing cathedral example.
Bazaar Benefits
I still think the bazaar model is superior. A single person being both a user and a developer of a product is ridiculously powerful. This is being taken advantage of in a world where the percentage of people who know how to code is increasing.
Bazaar Problems
There are problems with the bazaar model that seem unfixable and relevant.
The first, and most major, is that the bazaar model doesn’t mind failure in the short term. OS scheduler is buggy? Totally fine. University database accidentally deletes all data on a student? Annoying, but fixable. Failure is great in projects that can fail forward and use software evolution to cull the worst mistakes.
However, failure is unacceptable in some cases. If software is being written for rocket guidance or patient monitoring in hospitals, the tolerance for failure is orders of magnitude lower than a custom OS running on your laptop.
Bazaar development might still succeed in developing robust software in these areas, but could lead to significant short-term costs in money or life. Alternately, any form of sandboxing or initial testing without risk would be heavily encouraged. For example, the Falcon 9 rocket was stress-tested in the form of numerous cargo flights and intentional limit-pushing exercises. This move-fast-and-break-things development lead to much failure in the short term, but ultimately delivered a robust product once the design was frozen that now ferries astronauts to the ISS.
Another counter to this problem is that software always has bugs. One could argue that the bazaar model is more ruthless at finding and fixing bugs, leading to more robust software in the end. This might bias high-cost and life-critical systems towards bazaar models in the long term.
The second, and more minor, problem with bazaar development is that it relies on passion, interest, and irritation to spur development. This is a poor model for software that 1) is important, and 2) doesn’t affect, interest, or irritate a lot of people. For example, water treatment plants have numerous steps to take untreated water and make it drinkable. If the plant is running correctly, nobody notices or cares, but it’s important that software is correct. I would predict the bazaar model would fail to deliver on good control code for the plant, but a cathedral model with clear parameters and monetary compensation would do a pretty good job.
Conclusion
Given all of this, there is absolutely a place for cathedral development in the world for a small subset of problems. However, I am much more excited to live in a world where 1) passions are strange and varied, making sure just about every project has some passionate dev base, 2) problems are shared and discussed on an international network connecting everyone, and 3) more people can understand and fix software problems than ever before.
Notes
A Brief History of Hackerdom
- “Your historian first became involved with the hacker culture in 1977 through the early ARPAnet and science-fiction fandom. From then onward, I personally witnessed and participated in many of the changes described here.”
- “C was designed to be pleasant, unconstraining, and flexible.”
- even modern OS’s like linux aren’t written in assembly, they’re written in C
- c is the abstraction bridge to assembly
- will there be another higher level?
- bazaar → evolution comparison
- great for rapid, error-prone dev
The Cathedral and the Bazaar
- substance from chaos
- “grace and dispatch” when relinquishing control
- the transparency of a system increases with the number of users
- brilliant: separating the idea from the understanding
- simple is better, ask the right questions
- robust and simple better than cute and complicated
- model doesn’t work for things that should get built but aren’t interesting
- counterpoint is that given enough people, eventually someone finds a problem interesting or irritating
- i’ve always felt bad about not coming up w more new ideas → not a problem if I can identify/understand good solutions from others
