One Piece Flow

Many people don’t realize that Lean Software Development is based on the Toyota Production System (TPS). Toyota’s way of engineering cars is radically different than how most companies build cars. Instead of batch and queue (i.e. mass-production assembly-line), their manufacturing process is optimized for one-piece flow and pull-processes. Toyota’s way of building cars has demonstrated to produce higher quality products, increased productivity, faster time to market, and happier more accountable employees. Whereas assembly-line mass-production often results in over-production, lots of waste, and demoralized employees.

Achieving one-piece flow is about figuring out how to create a single quality unit as fast as possible while minimizing inventory, wait-time, transportation, hand-offs, etc. Through this process, the overall time from beginning to end is minimized to contain only all value-add steps. When there are defects in the manufacturing, the entire process must halt and the problem immediately fixed to ensure quality is built-in and there are no defects in the end product. Many organizations have problems implementing a true one-piece flow, it requires constant improvement, or kaizen.

Continuous Integration & Continuous Delivery (CI/CD) is the Toyota Production System’s concept of Flow applied to software development. At Toyota, the one-piece they’re making is a single car. In software, the analogous one-piece is a deployable software feature/task. Assuming trunk-based development and a modern containerized CI/CD setup, a developer typically writes code, runs unit tests locally and then commits it to a pull-request. As the pull-request is committed against, the new code is continously integrated and tested by the continuous integration system (i.e. built-in quality). After the pull-request is reviewed by a peer it’s permitted by the system to be merged into the master branch. At this point, a container is automatically built by the build system. Once built, the container is sent to a staging environment, and service tests are run against it. If they pass the tests, the container is blessed and deployed to production. Deployments to production must take minimal time but be fail-safe (i.e. blue-green), if they fail, they must rollback automatically. This process is a true one-piece flow, the build contains only the feature that is asked. If there’s something that breaks during any of the processes the developer is right there to fix it immediately. They don’t move onto another task until the feature is deployed and running successfully in production, they stay engaged the entire time.

There are pitfalls that people sometimes fall into while developing software. Sometimes people bundle multiple features into that deployable one-piece. Batching actually increases risk, when there’s more new code it increases the chance that something is defective. When something is defective, it now requires rolling back the batch of features instead of the single feature.

Sometimes people cut corners, they don’t write unit tests, integration tests or service tests. This is a false economy, you can’t have built-in quality if you don’t employ the right processes and safety nets. The engineers at Toyota say, “the right process will create the right results”. At Amazon.jp warehouses, they have a saying plastered everywhere, “safety is #1”. The same concepts apply to software development. The tests are the safeties.

There’s a lot of tooling that needs to be invested in to make one-piece flow work, sometimes people don’t spend the time investment to put the right machinery in place or continue to improve to make the processes more efficient (reducing waste). Good hygiene and exercise is necessary if you don’t want to get slow and fat.

Software can be complicated, the process of developing software doesn’t need to be.

Amazon HQ2

The Toronto tech scene is hot.  Last year, more tech jobs were created in Toronto than San Francisco, Seattle, and NYC combined.  By making Amazon’s HQ2 shortlist, Amazon essentially vetted Toronto as a great tech hub.  Amazon saved all these companies thousands of hours of research, cities put pitch decks together for all to see, and now tech companies are moving in fast.

In the past few months, Bay Area startup heavyweights and big tech-behemoths have announced they’ll be recruiting large engineering teams in Toronto. Uber announced they’re investing $200M to build an engineering hub in Toronto.  Microsoft announced they’re moving their Canadian HQ to Toronto. If Toronto becomes the site for HQ2, we should rename Toronto to Seattle2.  Amazon already has over 700 tech employees in Toronto working on Alexa, Supply Chain, and Fulfillment systems.  Their footprint here is already sizeable.  The Bay Area heavyweights are all talking about building 200+ person engineering teams in Toronto.  

This will have both positive and negative effects for sure.  It’s going to drive up the cost of talent, poaching talent away from promising startups to join more established companies.  I honestly believe that life as a software developer is vastly different at Amazon vs a startup.  But companies like Amazon and the more established startups serve an important part of this ecosystem, they are finishing schools for new grad talent.  I’m convinced after they’ve worked a few years in a big-tech company, they’ll itch to join a startup and have more impact.  

In the short term, the biggest losers will be early-stage startups that are trying to find product-market fit and have limited funding because the loss of talent will be felt most acutely. Time will tell, but I believe these developments are good for Toronto, but there will be growing pains.