Of Software Development and Saving the World

I heard a story on NPR today that made me connect the failures of software development and the failures of NGOs and other attempts at saving the world. The problem almost always comes down to scope and an inability to deal with change.

Today's story was in part about "Mango Man" a Haitian business man attempting to sell mango's to New York grocers. His idea was that if he could get more high quality mango's from local farmers then he could double their, and his, profits. The problem seemed to be a simple problem with a simple solution.

Mango farmers often didn't clean the fruit, mishandled it, stored it improperly, and transported it haphazardly. This resulted in slightly damaged produce that the American market doesn't tolerate. "Mango Man", Jean Maurice Buteau, devised a simple plan, give the local farmers plastic crates to carry the produce in and encourage them to clean the produce. So to that end he traveled around handing out plastic crates to local farmers he happened upon.

It ended in failure.

Why? We'll get to that, after the second part of the story.

Buteau found that the farmers misused the crates, lost them, broke them, and generally didn't know how to use them properly. However, he still felt that his plan was worth while, he just needed help with the farmers. So he engaged a friend at an NGO he was familiar with. This is where the simple project turns into a not so simple project. The NGO says that the crate idea will never work. Based on their experience the best thing to do is to create a coop, centralize the farming to a shared piece of land and ship the produce in bulk.

So begins the logistical nightmare of finding land, fighting family members, getting farmers buy-in, transferring deeds, budgets, budget re-estimation, getting government authorization, forming a legal cooperative, and over a year of work. Then there was this little earth quake that leveled the building holding the documents that recognized the legal cooperative. All fairly well detailed in the NPR podcast here http://www.npr.org/blogs/money/2010/02/podcast_dreaming_of_plastic_cr.html.

Then after the earth quake someone decides that maybe the quake is actually a boon, because now we have 10,000 NGOs combining their efforts instead of doing one off work. Plus we have the focus of the whole world pouring money into these NGOs and volunteers showing up and more resources and more people and more attention and more... So the attention starts to turn to how to solve the bigger problems like roads, because how does the farmer get to market over dirt roads. Then another problem occurs, how is all that produce going to ship once the farmer has brought it to market, it might overwhelm the small port, so we need a bigger port. Then another problem occurs, a bigger port costs money so we'll need a better bank and better banking infrastructure. And so on and so forth.

So why is this story like software development? Well... you have three classic cases of failure.

1. Project 1 - Deliver crates - Fails to take into account the users of the crates. Fails to instruct them or reaffirm the proper use or benefit of the crates and therefore fails to improve the quality or quantity of product and in return fails to improve revenue.

2. Project 2 - The Co-op - The project takes a turn toward bureaucracy. It has all the makings of a standard waterfall development effort including huge upfront planning, set and restrictive budget, and an inability to adjust to change easily. The focus of the majority of the project is on paperwork and after over a year the principle stakeholders have failed to meet the basic objectives of increased product quality, quantity, and increased revenue.

3. Project 3 - The Whole Shebang - Obviously the other projects fail due to lack of getting the big picture! So the real problem must be one of infrastructure. Problem A leads to B, B to C, C to D, and pretty soon your produce quality problem has turned into "how do we upgrade the local airport and hotels so more celebrities can visit in order to endorse eating mangoes." This is classic scope creep and over-engineering. It also exposes the typical "throw more resources at it" mentality.

Nine women can't make a baby in one month!

These problems: failure to understand the user, focusing on the paperwork instead of the product, scope-creep, and increasing complexity are all common to software development and trying to save the world. Countless NGOs and GOs and NFPs are running around the world attempting to solve the problems of an area through careful and meticulous planning, assuming that such planning will help them overcome real-world obstacles. To some degree that's accurate, they do overcome real-world obstacles. But those obstacles are not the root cause of their problem.

In the story the NGO assumed that the only correct way to face the problem was to create a coop. This solution then presented it's own series of challenges from real-world obstacles. The only solution was seen from the point of the NGO, who had seen numerous other failures in the past and their successes were only in the areas were coops were created. However, how did they measure success? Was success measured in that a coop was created? Was it measured in the fact that higher quality produce was provided from those coops? Was it measured in lives affected or increase in average income? We don't know. Likely the only measure was based on what brings more volunteers and more money into the NGO, because more resources means more can be done, right?

Wrong!

NGOs aren't bad. They do a lot of good work. But like software development they can go way off track and end up in bureaucracy and scope creep all too easily. While some might look at this as "the real world" I've seen and read about cases that lead me to believe that it's just wrong.

Take for instance Scrum in software development partnered with User Centered Design. You have some core tenants that help you avoid all of the problems of the three projects mentioned in the NPR story. Some of these tenants are:

1. Know thy user
2. Know how to eat an elephant - bite size chunks - (smaller efforts are more often successful)
3. Fail quickly/Recover quickly
4. Observe and obey constraints
5. Delivery early and often
6. Focus on quality not quantity

So we can use those tenants to rethink Mr. Buteau's original problem.

Mr. Buteau's goal was to double his profits by providing more high quality mangoes to New York.

Problem - how to get more high quality mangoes?
Constraint - Buteau's own supply was not sufficient and there were no other farmers large enough to provide the quantity needed.
Solution - Leverage multiple local farmers to increase supply of high quality mangoes.

Problem - local farmers mishandle, improperly store, and ship the mangoes haphazardly.
Constraint - low education and socio-economic status of the farmer.
Solution - provide consistent container for farmers to collect, store and ship the produce in.

That's where Mr. Buteau stopped. He went and got the containers. Drove around to farms. Handed the containers out and then expected things to roll on from there. He failed to take into account the user, their motivation and their cultural beliefs.

The user in this case saw no reason for taking care of the mango. To them a bruised mango was just as edible as a nice mango and sold just as easily at the local market. The container that was provided was no more useful than any other container they had and in some cases was more useful as a book shelf at the local school, or a chair, or a footstool. So at the end of the day the containers were lost, broken, or otherwise unused for their original purpose.

So then we move on to another problem

Problem - local farmers need to be educated on the proper handling, storage, and shipping of the produce
Constraint - average farmer education level, distance between farms, lack of training facilities, lack of budget.
Solution - This is where the NGO decided to move toward centralization into a coop. Centralizing would make it easier to educate farmers and insure consistent product. But is it the only solution?
Solution - constrain the number of containers being provided, use a portion of the budget to have printed instructions on the side of the container or on a laminated card securely attached to the container. The printed pictographs should indicate the steps to a quality product. Example steps:

1. Show fruit being cleaned
2. Show unspotted fruit going to America (double $$ sign)
3. Show level to which container should be filled and not filled (overfilled)
4. Show pickup/drop off point for full container
5. Show pictographs warning against improper use

Show alternate instructions showing similar steps with spotted fruit with last step being representation of local market.

The goal would be to show the end user the proper use of the container without requiring significant training as well as help them to understand the benefit of cleaned and undamaged fruit (e.g. more money). The benefits of the instructions on the container means that if the container changes hands to another farmer or family member then the training goes with it.

By using a portion of the budget for the crates to create training material and thus reducing the number of farmers the containers go to Mr. Buteau could be more selective about which farmers to engage. For example, picking ones he had already seen higher quality product from, ones with more education, or ones he had cultivated a relationship with. These lucky few become a test bed for what does and does not work allowing future rounds of container outreach to be improved.

Mr. Buteau can also offer at-cost containers to other farmers as word spreads. If one farmer using the provided container upgrades their house with a better roof or is able to send both of their children to school instead of just one, other local farmers may learn of the change and want to follow suit. In this way the original small contingent of farmers given free containers can become mentors to other nearby farmers without the need for centralization.

If Mr. Buteau finds that not enough produce makes it to market unscathed due to poor travel conditions then that's a problem to be solved when the need arises and not before. In that case it might make sense to set up local regional drop-off/pickups so that product isn't transported as far under poor conditions. This is where middle men may come into play and eat into the original profit some. However, this also adds to the local economy creating jobs and income and with income comes more educational opportunities.

While Mr. Buteau will not be able to increase the quantity of product or increase revenue immediately he should be able to reap some reward right away and begin to scale to the desired level over a short period of time as he shows the benefits of his container solution. This may also mean that for a period of time he sticks to secondary markets until the quantity and quality of the product are raised to sufficient levels to be ready for a city like New York.

There were a variety of other solutions in the comments from the NPR site including comments about dealing with government corruption, the need for Mr. Buteau to just outright reject substandard produce wholesale, and the need for bureaucracy as a means of maintaining standards and order. None of those deal with the day to day reality of a Haitian mango farmer. The farmer isn't worried about government corruption or standardization of the product or process. They're worried about how they feed their family, how they get their children educated and get them a better life, and how they can keep a roof over their heads.

If Mr. Buteau rejected produce as being substandard it wouldn't immediately make the mango farmer think "I need to improve my efforts" it would more than likely make them think "I've just wasted a whole bunch of time and I can't afford this. Who's this light skinned jerk think he is, I'm not doing business with him ever again" and then they'll go tell everyone they know what a rip-off the whole system is. That benefits no one.

Feel free to poke wholes in my theory or add to it. I'm not going to claim it's perfect but I think it does attempt to look at the solution from a different perspective that may scale better and provide value faster than the whole shebang ideas that were out there.