Random Thoughts on Community Mechanics Relative to Open Source Project Size
When you start an open source project, it’s yours, and it’s usually super fun.
There’s a direction set, and you probably will gather a degree of people (if you are lucky) that think similarly. You feed off that energy.
As it grows, ideas mix. You’re given access to some great sounding boards.
As it grows further though, the direction one person wants to take things begins to conflict with where it needs to go. How do you communicate that you aren’t doing X, because there are 50 different more important things you can be doing and you don’t want to forever maintain and support the code for X?
Even in the first months, you quickly will transition from having a way to express creativity to helping a lot of users on the internet for free. What do you do, and how to do you continue to keep it fun and interesting?
There’s a nice quote from Star Trek II though, that applies. ”The needs of the many outweigh the needs of the few, or the one”. This is the philosophy, I think, that works best in large projects.
A frequent example of complaint is the Linux kernel. While it’s easy to say it is managed aggressively, there is a need for expedience. If every interaction takes 30 minutes, there can only be 72 — realistically 36 — meaningful interactions in a given day. And this is problematic — there’s a large class of folks that assume efficiency is being impolite, and it almost never is. Reading between the lines is a danger on the internet, but it’s an unavoidable curse of the medium.
It becomes vitally important then, as a community grows that users help users and become self-sufficient.
A primary enabler of this is documentation, as well as frequent communication, for which things like a project mailing list can be a great way for people to understand the project. (Though a recent trend in open source is for folks to just submit changes on GitHub prior to seeing if they are a good match — which works 50% of the time, and leads to confusion the other 50% — at worst it makes for a lot of expensive 1:1 vs 1:N conversations).
As a project grows though, you add new folks who don’t remember the goals of the project. While technical documentation increases, there’s a greater need for philosophic documentation — why things are the way they are, the way the project thinks, etc.
Yet, there’s a cardinal flaw in this — by and large, documentation won’t be read.
A recent example of not understanding the goals of a project was a project called neovim, which seeks to rewrite much of vim — perhaps (IMHO) the world’s best editor. Not only is the task likely not well estimated, it tries to do such to fulfill a fringe feature need that is not well aligned with the main project — it neglects the needs of vim’s userbase and the way the project has changed to deal with the scope and magnitude of what it now entails.
Another commonly cited example is “Innovator’s Dilemma”. The “New Hotness” can arise every N years, burn the forest, and start anew, and this is very common in technology because as things grow in scope, the scope becomes a larger thing to maintain.
The way to enable agility in the face of innovator’s dilemma - and to not get sidetracked by serving fringe needs — is to occasionally say no, and well define the goals and purposes of a project.
The easiest track when faced with contributions that don’t understand the purpose of the project — or the current priorities — is to do everything that comes in. Even telling every new idea about *why* you don’t want to do X can take a lot of time. But it’s important that “no” is said more frequently than “yes”.
As a species, we’re not a very good Hive Mind. We do well at recognizing patterns, but if you get 500 people to tell you what they want in a car or truck, you are going to get a very ugly car or truck that might only please 5 of those people.
When a project merges every single idea, without validation or thought, it grows to lack identity and cohesiveness, and soon grows to lack stability.
To handle this, a project must continue to know what it is — and what it isn’t — and concentrate on it’s core use cases.
In phases where it recognizes more time on X is needed, it must not spend time on Y, and must communicate that it is disinterested, at least for now, in Y.
And to be respectful of incoming change, it must communicate when a request comes in for Z, if it’s likely never going to be interested in Z, that it’s likely never going to be interested in Z.
This itself can be a source of conflict — especially as, many uses believe open source projects are more of a playground, when the reality is they are designed to solve specific problems.
The question that should go through the mind of a maintainer, every single second is, where can I apply the most effort now to do the most good.
"Hey look, a squirrel" is an easy phenomenon to get into. Does the most recent request or question get the most attention?
What you have to do is sample the entire swath of a userbase, understand all of it, and declare focus to solve the problems that affect the most people first, and avoid distractions.
So there are people that might say, poke fun at Linus Torvalds for the way he might reply to someone on a mailing list. While I’ll never do that — I get it — he has a hell of a job.
What project leaders are are arbiters of change and priority — with a sufficiently large project — it’s impossible to service everything at the same time, and importantly — it’s important that a project not go in every direction at the same time.
On one end, you spread yourself too thin, and to an extreme, there’s being Drawn and Quartered.
I think Linus is doing a heck of a job.
Especially when there’s a chance for something to be misinterpreted, coming off clearly and without ambiguity is difficult.
It’s not a level of force I’d ever wish to see in most projects, but in the kernel, it may be warranted. Like Spock said — the good of the many outweigh the needs of the few, or the one.
If trying to please everyone resorts in pleasing no one, identifing the use cases you want to solve, and solve them well, becomes important.
And if you’re not going to solve the other use cases, it’s important to let people know, so you don’t be caught up explaining them infinitely.
So yeah, the kernel list can be a fun place, but I understand why it is that way completely. It’s building a great product, and you can’t argue with that success.
I guess what I’m saying here is that everything that has ever been written about how to run a good project changes drastically as that project scales — and there are very few people that have had that level of experience. There’s an entirely different continuum and it feels altogether different.
While everyone would like a democracy, there haven’t been any in a very long time (even in Greece, not everyone could vote). Representative republics (aka the USA) are notoriously inefficient. Google “Twitch plays Pokemon” to see how that works most of the time.
Avoid turning your project into a free for all — “Wikipedia with pull requests”. Don’t be afraid to be bold and define your direction well, and make sure you still have time to do the things that need to be done.