When you think of a pipeline, what comes to mind? If you’re like most people, it’s probably oil and gas. Companies that extract these natural resources out of the ground need to transport them to refineries, production facilities and so on. A literal pipe needs to be in place to get the oil and gas from one location to another.
The pipe ensures the material gets from one station to the next from extraction, refinement and ultimately on a truck somewhere to deliver to the end customer. The pipeline makes it possible to get this material to every place it needs to be and the entire
process cumulatively purifies the product to an end result we fill up at our gas stations and heat our homes.
You may be asking yourself, “What does an oil and gas delivery process have to do with me? I’m a knowledge worker that sits in front of a computer all day”. You’d be surprised!
Think for a minute how work gets done; there are an input, processing, and output, just like a computer. New projects come in like a new marketing campaign, a new training course needs to be created, new employees need to be onboarded, etc. The list is endless. There’s always another project that needs to be finished. That’s the input; the need to accomplish something. The processing part is where the rubber meets the road; it’s where the work gets done. Then finally, the output is the desired outcome like 1,000 leads were reached out to, that online learning course was published or that employee now has the handbook, a new computer and has gone through new-hire training.
Ready to stop reading and start learning about PowerShell, DSC, Windows Server, Sharepoint, IIS and dozens of other categories? If so, check out the hundreds of free technical demo screencasts available on the new, IT career development platform TechSnips.
Each of these projects has a flow. This is why you’ll sometimes hear the term workflow. Work in of itself is a never-ending stream of inputs, processing and output. A workflow sounds a lot like that oil and gas pipeline, right? Stuff goes in, it gets processed and stuff goes out. When work is thought of in terms of a pipeline, you’re able to standardize, automate and get more work done. That’s what it’s all about!
Understanding Units of Work
As with the oil and gas pipeline example, the “work” that gets transported from dirt to gas station is oil and gas. Oil and gas can be defined as “units of work”. These materials are what we’re working with and what will end up in my car when the oil has made its way through the pipeline. For a knowledge worker, this unit of work is a little harder to define but not impossible.
Think about a car factory. A car is not a car when its metal skeleton gets put on the front of the production line. A car is a car, to you and me, when it comes off the production line, someone gets in the car and drives it off. A car is the end result of the work performed and, for simplicity, can be called a car whether it looks like one or not near the beginning of the process. A car, to an auto manufacturer, is a unit of work.
Tesla keeps numbers on how many cars were produced per hour/per day/per month. They don’t report on how many seats were installed or how many screws were added. Seats, screws, the engine and so on are all components that make up the unit of work; the car!
All projects have a unit of work except some are harder to define. If you’re running an email marketing campaign with the goal of getting email replies from 5% of total leads, your unit of work might be a single company. Each company on that spreadsheet must have the same set of actions applied to them.
- The contact person at the company needs to be found
- That contact person’s email address needs to be found
- An email needs to be sent to the contact person
- Whether or not the email was delivered to that person needs to be tracked
- Email follow-ups to that person need to be performed if no reply
- ..and on and on
A certain set of tasks must be performed on each “unit of work” or company.
Stages of Work
Work, as I just mentioned, can be broken down into three phases or stages; input, processing, and output, however, we need to get more granular to truly leverage this mentality in our day-to-day work. Just about every project or task can be further broken down into various stages with units of work flowing through each stage. For example, perhaps you’re a software engineer in a software company and a new requirement comes across your desk by management to implement a new software feature. You now have your unit of work; the change in the software-defined as a feature.
Let’s say, for the sake of argument, that you have all of the information you need to get started (input). To implement this new software feature requires building a server to test on, writing the code, testing the code, sending the new feature to a QA team where they test the code and then finally it’s rolled out to customers. Applying work pipelines in this context is referring to as a build and release pipeline in the software industry.
Software companies know that coding can get complicated. There are lots of areas where bugs can creep in and problems can arise. Engineering teams use to build and release pipelines to automate and streamline much of this process by breaking down each action into various stages.
The stages in a typical software release may be:
- test server creation
- test software installation
- new feature integration
- engineer feature testing
- QA feature testing
- feature rollout
The pipeline or workflow is segmented out into six distinct stages. In each stage, a standardized set of processes must take place before the work can be progressed to the next stage. In the software community, this is sometimes called the value stream where value is being passed from one stage to another.
If you’re not in the software industry, the concept of a work pipeline will still apply. Every project or task can be broken down into smaller tasks no matter the size. The work done in each stage builds upon the previous getting the work closer and closer to the end goal.
How to Implement a Work Pipeline
As alluded to earlier, defining work as a pipeline with stages has a lot of benefits. The very act of defining work as a pipeline has it’s own benefits even if the pipeline approach isn’t even used! By simply being forced to think about your work in this manner will naturally lead you to understand your process better and will help you see the flaws in your approach. If you choose to go ahead and set up a work pipeline, the process improvements can be extraordinary.
Benefits such as automation, business intelligence, and standardization can all be achieved by defining work as a pipeline.
To demonstrate these benefits in a real-world scenario, I own a company called TechSnips. This company produces short screencast videos (unit of work) explaining how to perform various technical tasks in the information technology space. One metric the company uses to measure success is how many videos are published every day. We know that the more videos we publish, the more successful the company will be overall.
We define our work pipeline in Trello. Trello provides us with the ability to track work with cards which represent a single unit of work. We also define each stage as a Trello list going from left to right as you can see below.
A card cannot proceed to the next stage until the required work has been performed on it in the previous stage. Once that is done, the card is then dragged to the next Trello list and on and on until it finally gets to the To Publish stage.
In each stage, we use Zapier to trigger certain actions based on the stage that a video is currently in. This allows us to perform a consistent, predictable set of actions on every video we publish for our customers. This process greatly reduces human error by not adding a video description, introducing a typo to the video title, uploading a video that’s too small, too big, etc. Automated testing is performed throughout the entire lifecycle of a video.
A huge benefit for us is automated testing. We are able to publish better quality videos because this testing is automated and mandatory.
Another benefit of our work pipeline is for tracking and reporting purposes. One of the tasks that trigger when a Trello card is moved to a new stage is a sync to AirTable. We keep a database of all videos already published and are soon to be in AirTable.
This provides us to see, at a glance, various attributes related to each particular card. For example, in the screenshot below, you see one video that has no Record Start Time or Completion Time because no one has started recording it yet. However, the other videos have these attributes because they’ve already been published. Each video inherits different attributes like this as it flows through our pipeline.
Other actions in our pipeline include automatically uploading videos to Amazon S3 when submitted for review, publishing the video when complete to our video platform, submitting to clients (only if necessary) based on pre-defined rules and more.
Since we have defined our unit of work, laid out the stages and set up the tasks necessary to perform at each stage, it’s now simply a matter of following the rules and moving cards. We’ve leveraged automation to eliminate many potential problems and have boiled down this complicated process to simply checking off boxes in a checklist!
The End Result
You have seen that by defining your work in terms of pipelines or workflows frees you up to do more creative work, ensures better quality and greatly reduces human error. Although potentially time-intensive up front, building a work pipeline is a hugely beneficial endeavor that will save countless hours of time and frustration down the road.
My company and all of the other examples I’ve provided here are just that; examples. The key is to figure out how to apply this principle to your situation. With a little creative thinking, ingenuity and time, work pipelines can benefit every individual or organization.
The key to getting the most out of a work pipeline is the tool. My company chose to use Trello as a tool to implement our work pipeline. The tool doesn’t really matter. As long as you’re able to break up your projects into units of work, define stages, set up tasks associated with each stage and have a way to move work from one stage to another, it’ll work out great!