The Development Process

The development process has been extensively researched and there are many processes or methodologies that have been proposed over time. This document attempts to identify simple concepts that will be generally useful.

The Role of Documentation

Documentation plays an important role in development. There are two ways we can approach documentation:

1. As a set-in-stone specification that is used to drive the implementation of the project. In this model, senior developers or architects write the specification, and any changes to the specification need to be made by the original authors of the document. By the end of the project, the documentation is usually out of date and may bear little resemblance to what is actually produced.
2. As an active, living document that changes as the design changes and is used as part of the design process. As requirements and conditions change, the document is used as part of the thinking and collaborative process. When new features are starting, the approach is thought through and briefly documented before implementing. As the development proceeds, additional notes and changes are made to the documentation to reflect the approach, why things were done a certain way, etc.

These are perhaps extreme examples and perhaps reality lies somewhere between, but they do illustrate a point.

The Development Triangle

Life is full of trade-offs – product/software development is no different. Fast, Cheap, Features … pick two. Want all three? Then quality will suffer.

Why is Iteration important?

There are several attributes of modern systems that are perhaps different than systems in the past:

• They are often complex such that no one person understands the entire system, or can even visualize all the possibilities. The most innovative and useful systems are a combination of many good ideas.
• There are plenty of resources for expansion (memory is cheap, processors are powerful, standard interfaces like USB allow for easy expansion)
• The system can be upgraded (software updates are standard)
• The problems they solve are complex and the problem often changes once people try something and better understand what can be done.
• Perfection comes from refining ideas, trying things, etc.

The software industry has long understood this and most development processes have moved from a waterfall to agile type methods. However general project and hardware development thinking tends to lag -- partly due to practical considerations such as the cost of building physical prototypes. However, these costs are becoming lower and lower each day and innovative companies have embraced iteration. As this article states so well:

Apple does this with their hardware. Why don’t you?

Business Relationship Models

The simple way of managing projects is the "throw it over the wall model." In this model, you typically have a gatekeeper that manages the over-the-wall transition.

This model works fairly well for simple systems or components that can be clearly specified and no iterations are required. In this case, an experienced gatekeeper may be able to add a lot of value in managing the process. Gatekeepers may be a consultant, distributor, manager, purchasing agent, etc. However, as the complexity of a system increases, the gatekeeper increasingly finds they don't know everything about the system. Instead of improving the process, they become a bottleneck.

As systems become more complex, the need to collaboration and project iteration grows -- the success of open source software has long proven this. Thus having processes where all parties involved can easily collaborate is critical.

If open processes are not in place, the following happens:

• the best ideas are never discovered, as the right people never work together
• problems are extremely difficult to solve as the right people don't have visibility
• motivation suffers because not everyone gets to share in the project success or failure
• things move extremely slow

Thus, for modern systems, it is important to foster processes and environments that encourage openness, collaboration, and iteration.

Optimize where it will make a difference

A comment we sometimes hear when we ponder approaches is "Just keep it simple!" The question we then must ask is "from whose perspective?" With any optimization or trade-off, we need to always ask this question.

A few perspectives to consider:

• R&D
• time to market
• maintenance
• support
• end user
• sales
• manufacturing
• quality
• management
• financial
• business owner
• IT
• legal

It can be a little overwhelming when you consider how long the above list can be. One approach is to consider who is spending the most time with the product. Hopefully this is the user, otherwise the product is not successful. If you don't optimize for the user's experience, then someone else will. The next categories might be manufacturing/maintenance/support/sales -- at least for a long lived product. Sustaining development effort far outweighs initial development on most products. Additionally, sales, manufacturing, and support efforts continue as long as the product exists. The time developers spend working on a product swamp that of management and support functions like IT. We obviously need to balance the concerns of everyone involved, but if we optimize the tooling and flow for those who are spending the most time with the product, that will probably pay the largest dividends.

Another approach to consider is try to implement tooling and processes that benefit multiple perspectives. Some ideas to consider:

• select tooling that is open and can be customized as needed. Too often proprietary tooling is selecting from one perspective and it can't be easily adapted as new needs arise.
• ensure access to information is open. For example, if management has clear visibility and access to development workflow, then less meetings and manual reports will be required. If development has easy access to support and user information, they will do better job building something. The more easily information is shared and organized across organizational boundaries, the better things will go.
• automate - automate - automate. If something needs done more than a few times, automate it. This requires prudent technology selections that are amenable to automation and customization.
• remove unnecessary roadblocks and gatekeepers. If something needs done, make sure anyone with the skills can do it. Mistakes are costly, but not getting things done is likely more prevalent and more costly. The difference is the first is highly visible, and the latter can be hidden quite easily.
• focus on effective measurements that are visible to all. It's hard to improve without measurements.

Attributes of Successful Teams and Projects

Below are some of the attributes of teams/project culture that contribute to success:

• intrinsically motivated people (you have to start with this)
• very clear goals (with milestones)
• clear list what needed done and who was assigned to what specific task
• good communication
  • communication is centered around documentation
  • even though it is distracting at times, the best projects used instant/group messaging (skype, hangouts, signal, slack, etc) quite a bit. This seems important for relationships.
  • good use of issue tracking system (Trac, Github/Gitlab/Gitea issues, Trello, etc)
  • occasional face to face meeting (perhaps once or twice per year).
• developers get feedback on how what they are building is used.
• no politics.
• freedom to express thoughts and respectfully disagree with others without fear of repercussions
• no turf or rigid roles -- people did what needed done, and did not worry about stepping on someone else's toes
• openness - state of the project, how company was doing, field successes/failures was clearly communicated to all on the team
• everyone on the team had access to the same project resources, source code, etc.
• no barriers to communication -- you communicate with anyone on the project (even customers at times) and did not have to funnel information through a PM. However, because of the workflow (issue tracking system), a lot of communication (especially status and decisions) was also accessible to anyone on the team as needed.
• no blame culture. Mistakes will be made. Focus is on improving process, testing, etc. instead of blaming people when things go wrong. (Note, this does not excuse lazy or unmotivated people -- that is a different problem than making mistakes)
• low friction work-flow where experienced people review/approve work of less experienced people, but anyone is free to do anything needed.
• bias toward action, building, and early testing.