Programming Process Constraints / “Artists Ship”
Paul Graham (Y Combinator, the first spam filter, and the first web-based app: Viaweb) is one of the smartest guys on the internet when it comes to startups. He makes some great points in his article about placing controls on processes in big companies and small startups. His thesis is that controls always cost more, and the more controls there are, the more resources are taken up by them.
Given my limited experience (only eight years of working and one failed startup), typing up this disagreement feels a bit sophomoric. On the other hand, the entire focus of my career has been on setting up systems to manage constraints in a seamless fashion. In fact, I specifically work to establish controls in places where they can be automated and seen as goals instead of constraints. Attitude is important when you’re dealing with artists.
Defining an “Artist”
For the sake of this discussion, I’m going to define an artist as someone whose purpose for doing something is entirely in the heart, and whose compensation for performing at task is either entirely or primarily intangible.
Yes, you’ll find artists at all levels of corporations and organizations without regard for size or mission. You know you’re dealing with artists when you see them put that extra little bit in to make a project successful — and then they go home and continue to build out their knowledge and their body of work, and they’ll blog / tweet / contribute to open-source projects, and otherwise make their work indistinguishable from their life.
Organizations of Artists
Let’s take a step back and look at it from an entirely corporate point of view.
As companies grow they invariably get more such checks, either in response to disasters they’ve suffered, or (probably more often) by hiring people from bigger companies who bring with them customs for protecting against new types of disasters.
I would replace “grow” with “age” in the previous paragraph. Companies can stay the same size but integrate ideas from all over — conferences, employees that read blogs, new hires, training, regulation requirements — much the way that bacteria transfer genes that increase antibiotic resistance between neighboring bacteria. The best, most subtle ideas spread in a somewhat viral nature.
What do organizations made up of artists have in common? Generally, they create a lot of emotion. Look at Apple’s branding and marketing. It’s all emotional.
Placing Controls in the Real World
Case in point: I do volunteer work with the Impact Animal Foundation. Volunteer work with a noble cause like a no-kill, 100% foster, limited intake adoption program can leave people feeling good, but it can be extremely exhausting. Burnout is a huge problem for both the artists in a corporation and volunteers in an emotionally involved charity.
To keep people from getting burnt out, the organization has some pretty strict rules on what kind of animals they’ll take in, how many they’ll take in, how many shifts a week someone can work … all common sense stuff. They also have a process for making sure a home is right for an animal that involves people trained in animal and humor behavior analysis, a home visit, “play dates” with any other animals in the house, and a contract that specifies that if the owner can’t take care of the animal, that it comes back to our rescue and not a kill shelter.
See those bolded words? Those are the controls. Just as Paul pointed out, all of these policies also come from a situation when either the organization, an individual volunteer, or an animal was hurt in some way. They may cause the organization to not adopt an animal even though there’s a willing and loving home available. On the other hand: They ensure that the organization will continue to be successful even with the emotionally charged environment that it has to operate in.
Making Controls into Goals
The primary goal of the animal rescue is always met:
- volunteers go home feeling good at the end of the day
- every single one of the animals that the organization adopts out are happy, safe, and loved for the rest of their lives
Could we help more animals if we didn’t limit who is taken in, or were more free with adoption procedures? Yes. Could we always know, beyond a shadow of a doubt that the animals were in good homes? No. We’d lose fosters right and left because they’d hear horror stories of where their beloved temporary family members ended up. Could we always be sure that we weren’t stressing our network of fostering volunteers if we took in every animal that came our way? No, we’d lose fosters left and right as we placed unfair demands on them. The organization is based on it’s fosters. Without fosters, there is no animal rescue. The controls on animal intake and placement preserve the organization.
In the software world, a company’s based on it’s customers. Releasing broken code too quickly will send customers away. Releasing updates too late will allow competitors to get in the door — and will send customers away. Without customers, there is no company. Controls on software releases exist to preserve the organization.
If you’re doing it right, your artists won’t see constraints, they’ll see goals. The goal of the animal rescue is to take in the animals that are in great need (i.e. will be put down if they are not taken in) and are adoptable without an inordinate amount of work. The goal of the software company is to release software with no bugs on time. These goals serve controls, but end up serving the customer. The benchmark of a successful adoption placement is a completed application, a good home visit, and a happy union. The benchmark of a successful software release is passed tests and timely delivery. These are challenges suitable for artists.
Artists are competitive in ways that you wouldn’t otherwise suspect. Challenges add spice to what might otherwise become boring and routine.
Keeping Goals from Slowing You Down
At big companies, software has to go through various approvals before it can be launched. And the cost of doing this can be enormous—in fact, discontinuous. I was talking recently to a group of three programmers whose startup had been acquired a few years before by a big company. When they’d been independent, they could release changes instantly. Now, they said, the absolute fastest they could get code released on the production servers was two weeks.
This didn’t merely make them less productive. It made them hate working for the acquirer.
Yeah. Two weeks is ridiculous. If testing takes that long, well, you’re doing it wrong. No testing at all (immediate release) can also be a case of you’re doing it wrong — if you make a change that destroys production data, it could take you weeks to untangle … or worse, it could cause the permanent loss of that customer’s particular bit of data. Changes to production do need to be made carefully and tested. When you’re in “private beta” or “alpha” stages, customers may not expect to have their data be ‘permanent’… but once you reach a public beta release (especially if it’s backed by or been acquired by a large public company!), you’re held to a much higher standard.
Any place where procedures limit the artists from releasing needs to be evaluated. The only constraints on the artists should be completed and met BY the artists. Release testing is a good example. There was no release process when I arrived at my current job. Putting one in place was a headache until the (very passionate) developers finally “got it” that they could meet all the testing goals themselves, and they get the same sense of accomplishment by meeting them that they do from releasing an update in the first place.
Our developers can complete every stage in the production release process themselves except for the final release.
- Write Code, commit to Subversion.
- Write (and pass) Unit Tests.
- Write and pass Selenium tests
- Tag a release in Subversion.
- Do a “mock rollout” and write out any special instructions, such as database structure changes, for sysadmins.
When the release needs to go live, the Sysadmins quickly execute the above tasks. The data gets merged, the tests get run, and it gets rolled out. If the artists have done their job well, a change can take less than five minutes to go from development, through stage, and to live.
Everyone Ships
In our organization, it’s not the artists (developers) that ship — everyone works together to ship. The artists can clear it all the way through themselves. There’s a check/balance with the sysadmins. Ultimate responsibility lies with the managers. For a web development shop, we’re pretty large — six developers, two sysadmins, and about twenty to thirty projects or platforms.
I’m sure the same type of thing happens at Apple. The focus is on the artist, but everyone else is there to support the artist. In the case of the new MacBook design, a design engineer probably came up with the idea of using a monocoque frame. Did he submit the design to the purchasing department for them to find manufacturers? No, he probably either developed himself or went out and found a manufacturer capable of developing the processes they eventually used. Or he took it to his boss, who took it possibly up to Steve himself, and as a company Apple went and did it. Artists create, leaders lead, everyone ships.
Changes? They’re easy to sell, because they’re cheap. Processes that are streamlined so that they can be “pre-cleared” or “fast tracked” through? They’re efficient. Any restrictions? They make sense, and they’ve been taken to the leaders for clearance or confirmation. You can easily release in a few minutes even with controls in place. Letting artists ship does not mean throwing out your controls.
