The word serendipity was coined in the eighteenth century by Horrace Walpole based on his reading of The Three Princes of Serendip, a Persian parable of three brothers who through wit and circumstance end up receiving great riches. Serendipity is a word that means more than just a chance encounter. It is an encounter which creates fortune or benefit. And those who experience serendipity are actively seeking it. Therefore, they react to these encounters when they happen, taking advantage of the opportunity. In Steven Johnson’s book, Where Good Ideas Come From, he describes the liquid network, the primordial ooze that is the perfect blend of freedom and constraint that increases the likelihood that serendipity can occur. He explains how in the Age of Enlightenment, the English coffee house was a literal and figurative manifestation of the liquid network. In The Idea Factory, Jon Gertner writes about Bell Lab’s newly constructed New Jersey research facilities with a long central hallway designed so that employees would have to walk long distances between their offices and labs, “therefore making it necessary to walk between the two, and all but assuring a chance encounter or two with a colleague during the commute.” And in multiple publications about Steve Jobs, Walter Isaacson writes about how Jobs designed spaces like Pixar with central meeting points like the restrooms to ensure “serendipitous personal encounters” throughout the day.

For the most successful corporate innovation programs, serendipity is not something one thinks will happen, or hopes will happen, but is engineered and ultimately trusted to happen. For a well-designed program, the challenge is not that it will happen. The challenge is that there is no way to predict it. There is no KPI (key performance indicator) to measure it. And often, when it happens, there is no way to estimate its value until a long time after it occurs. And yet, serendipity will happen because successful programs learn to trust in it.

Since serendipity is such a big deal and one knows it will happen, then what again is the problem for the innovation program? The problem is that as the environment must be purposefully designed to engineer serendipity. This means there are choices and investments which must be made. And how does one sell snake oil like this to corporate executives? An environment where random, unpredictable, unmeasurable, unrecognizable encounters creating great benefit and fortune will occur?

The point is made in the asking. And it is not the only difficult and ethereal element of an effective innovation program. Beyond serendipity, there are 10 other reasons innovation programs struggle to be effective.

...serendipity is not something one thinks will happen, or hopes will happen, but is engineered and ultimately trusted to happen.

Reason #2) Innovation has real risk (and nobody wants to admit it). 

Before getting to the points about learning to deal with failure, and managing risk, and mitigating risk, let there be a moment of pure, unfiltered, honest, frank, real honesty. An effective innovation program is all about creating risk. If the ideas and projects being generated and managed by the innovation program do not create risk of any kind (legal, regulatory, financial, personal, professional, organizational, market facing, etc.) then please shut the program down and invest those dollars more wisely on things like shareholder dividends and employee benefits. Innovation programs must push on things (conventions, beliefs, markets, etc.) and what it pushes on can break, or in the act of pushing, the program itself may suffer the fracture. There are really smart ways to manage and contain that risk, regardless of the business, market, risk, etc. But do not fail to mistake risk management with outright risk avoidance. Businesses are designed to avoid risk. Innovation programs must be designed to find and embrace it. If one is inclined to question this point, then do not bother to create an innovation intended to create the next “Uber” (as an example) without acknowledging that Uber’s entire business model pushes at systems and structures that, for their business, create tremendous risk (legal, compliance, regulatory, etc.).

Reason #3) Innovation has real cost (and nobody wants to fund it).

There are only two things worth having which can be had for free, good weather and good advice. In both cases it will never be 100% good and in both cases the actual percentage depends on where one is and who one knows. What can be sure is that no effective innovation program can be built and maintained without an expense. Truly effective innovation programs will eventually generate value that far exceeds any other investments made in the organization (products, personnel, infrastructure, marketing, etc.). The challenge is that despite the value being generated, the bottom-line costs will always be scrutinized and evaluated independent of value. There are many reasons for this, the least of which is timing. So, like with risk, let there be a moment of honest and transparent truth. Innovation has a real financial cost. And despite all of the best techniques for maximizing the investments, and there are plenty of good tools for doing so, the simple truth is that good innovation has a real cost.

Reason #4) Innovation is hard work (and worse, creates even harder work for everyone).

In every organization there is hard work going on. At times that work gets really hard. Like auditing financials, rolling out a new performance management program, upgrading a critical software system, moving facilities, or acquiring a new company. Now take that level of hard work, multiply it by one-hundred, make it every day of the week, and increase the level of difficulty, not when success is absent, but when success occurs. That point needs to be crystal clear. Being successful does not make things easier, it makes it harder. It would be hard enough if all a corporate innovator had to focus on was understanding customer needs, product design, market size, financials, partnerships, marketing, and legal issues. But in addition to all of that, the innovator disrupts typical operations. So the hard work of the corporate innovator ends up creating hard work for everybody. IT, HR, marketing, product, finance, legal, leadership; everybody. Those risky and expensive innovations creep into every aspect of the business. And if the innovation sees early success and survives to scale, then it continues to generate extra work everywhere. And when a program is truly successful, it is not driving just one successful innovation, but many. And each of those successes creates more work for people who never wanted that innovation in the first place. So the innovation leader then has to work even harder internally. When people see the innovators coming, they turn and run away. They know what is coming and it is not a reprieve. There is a level of perseverance that an innovation program must be able to sustain and it is not the kind typically preached by innovation methods, the perseverance of market testing and validation. The kind of perseverance which is needed is the hard work of constantly and consistently pushing every aspect of the business to accommodate innovation in a way that the organization has intentionally been designed to not accommodate; anything different than typical day-to-day operations.

Reason #5) Innovation takes a long-time (and works on time-scales asymmetric to the business).

Organizations operate on the following time-scales: daily, weekly, monthly, quarterly, and annually. Many have three and five year plans and the ambitious (and sometimes unrealistic) have ten year outlooks. So it is understandable that new innovation programs begin with the goal of being fully operational in 3-5 years, including “creating a culture of innovation.” But the problem is that even programs that really seem to be successful even after 3-5 years generally admit that they are just beginning to get a handle on things. Governance is still being worked out. Tools and methods are still being tested and validated. People in the organization are just beginning to “get it” and feel like this “innovation thing” is actually going to be around for the foreseeable future. And after 5 years, successful innovation programs still feel like it could be cut at a moment’s notice. Any organization that gives a leader a mandate to create an innovation program in “2-3” years has already failed. Any innovation leader that thinks (and more importantly tries) to shorten the timescale will experience massive and damaging pushback. In cases, leaders think they can spend their way to success, endowing an innovation program tens or hundreds of millions of dollars as a war chest. Funding only solves one of dozens of challenges that an innovation program faces as it grows and scales. It does not solve the need for time.

Reason #6) Innovation only impacts a few employees at first (and may never impact the majority).

Today’s innovation programs are being deployed to engage all employees. The day of quiet R&D labs creating the next new break-through product in 10 to 20 years are over. “Culture of Innovation” and crowd-sourcing is all the rage. Now, the dismal reality is that only a small percentage of people in most organizations are naturally tuned to be “innovative.” The way they think and operate is already pushing the boundaries of the organization. They are the people who struggle to fit-in a go-with-the-flow without pushing on some aspect of the business that could be better. This is why an innovation program draws those early adapters like the thirsty to water. And they cannot drink enough. But it will take everybody else much longer to get on board. They will need to see real results. And those results, regardless of intent, funding, and hard work, will take tremendous work and time (#4 and #5 above). The great news is that there are ways to get people involved early on that is not “all-in.” The difficulty is to focus on those who have engaged. It is a common tendency in corporate initiatives to build for the least common denominator, or those who will engage last. It is not common to focus on a narrow segment of employees and slowly build around success with that group. Even more difficult is coming to terms that it will take an innovation program an extraordinary long time to meet the needs of a simple majority of the employee base. 

Reason #7) Innovation is unpredictable (and early plans and timelines are all fiction).

Today’s innovation methods all share a common belief in quick, agile, data driven development. This means decisions to proceed, quit, and pivot can happen at any moment based on the latest data point. An innovation program is as much an innovation project as any new product, service, or process improvement coming out of the program. Organizations are used to making plans and implementing those plans. All innovation programs will start as a well thought out, lucid, and executable plan like any other initiative. The difference is that the most successful innovation programs iterate, rapidly evolve, and pivot. The program’s plan at the beginning is absolutely a work of fiction, regardless of how well researched and designed. Any attempt to prevent the plan from changing will inevitably harm the program, not help. But the fiction is necessary. It is the baseline experiment from which all other experiments will evolve. The course changes might be perceived from the classic operator as a failure in planning and execution. In fact, those evolutions are the opposite. In a well-run innovation program, the evolution—deviation from the original plan—is the result of some planning followed by proper innovation program execution. Accepting the fiction as a necessary part of the journey will enable program leaders to make the difficult choices to deviate from the original plan despite the tendency and desire the hold the course.

Reason #8) Innovation requires trust (and trust is difficult to build and easy to lose).

When an innovation program is first launched, it will be perceived as the “programme de jour” or the “initiative of the month.” For those who want innovation to succeed, they will except the program’s existence unconditionally. However, their trust can be broken. And for those who do not believe this program will be around long, it will take a lot of evidence for them to believe in its reality. So much of an innovation program’s success is based on its ability to get people to trust. To trust in the process. To trust in the people. To trust that failure will be rewarded. And to trust that there is a long runway to get things off the ground. The reality of the situation is that trust is easy to lose and is difficult to build. Some people can persevere through mistakes and disappointments. Others will turn on the program in remarkably hostile ways. Some people will become directly invested and will therefore be personally disappointed at the slightest misstep. Like serendipity, an innovation program depends upon trust to be successful. As one innovation leader put it, “when we make a mistake it’s like you can feel them walking away.” And mistakes will happen. It is inevitable. This is why that once innovation program is launched, there needs to be strong and repeated commitment from leadership that the winds of market change will not lead to a sudden and fatal decompression of resolve toward the program. Innovation programs are a long-term bet. If leadership cannot commit “all-in” to this concept, then they will do less harm to employee morale, commitment, and trust by not launching the program in the first place than to launch the program and then pull back support before the experiment has run the course. It is critical to acknowledge and understand from the onset that trust is critical to success.

Reason #9) Innovation must embrace failure (and organizations are not built for failure).

From tee ball and kindergarten through college and one’s first “real” job, nobody is taught to “fail” on purpose, or frequently, or to embrace it. The idea of “failure” is so contrarian to most organizations that this becomes the most difficult concept to accept, and more importantly, embrace. Failure is so difficult to grasp that organizations allow massively expensive and ineffective projects and initiatives to exist for years because there is no analytical system to gage success and failure. Decisions to proceed with a project presume success in the act of approval. And there are such real political and professional consequences for failure that everybody does what they can to avoid it, including avoiding the unpopular and difficult decision to kill a project. The innovation community works hard at creating frameworks and vocabulary to make failure easier to understand or accept. This is most often done by explaining small cycles of failure as “experiments.” Spoiler alert. Failure is failure, regardless of how short or long in the making; regardless of how big or how small. And the reason to not shroud “failure” in any other term is because failure is always painful; the pain is cumulative; it never gets easier; and requires more perseverance every time it is experienced. Embracing failure means accepting the reality of failure; the emotion; the pain; the grief; the need for support and empathy; and the need to move on and grow through learning and acceptance.

Reason #10) There is NO innovation playbook (though one can be built for each program).                                                                                        

There is no corporate innovation playbook that can be transplanted from one organization into another or be delivered as a template by a consulting firm. There are several reasons for this but the most important can be articulated in just two points. First, the push for corporate innovation is still a relatively new approach. This means that there is not enough collective experience to draw upon to create universal models for corporate innovation. Look at approaches to enterprise project management (EPMO), Lean, and Six Sigma. One would be hard pressed to argue that any of those frameworks has a standard enterprise approach, and each is far more advance in terms of maturity and thought leadership. Second, organizations are a set of cultures and norms. At its core, an innovation program is not just tools and governance. It is a whole new mindset that requires entirely new ways of thinking and acting that is fundamentally different from ongoing operations. For many people, this will be terrifying. For others, this will be liberating. A playbook can be built. But it will require having permission to invent something that to some degree, whether its 10% or 90%, is unique to the enterprise.

Reason #11) Innovation has no corporate vocabulary (but there are ways to make it understandable).

There is a developing lexicon for corporate innovation that is distinct from corporate operations. There are two mistakes commonly made by innovation programs when the world of innovation and the world of operations collide. First, trying to run an innovation program through the lens of operations will drive innovators toward focusing on the wrong activities. This will lead to making the worst decisions because the desired results are being driven by the vocabulary and mindset of operations. Trying to generate revenue, profits, “return on investment,” market share, and so forth should all be long term objectives. But the corporate innovator at the onset of a project needs to keep the innovation objectives in mind: the acts of defining a core problem, validating a solution, using experiments and data to drive decisions versus emotion and bias. Second, the goal of a well-designed innovation program should be to allow the principals of corporate innovation to drive activity and define the innovation results which are trying to be achieved. However, these activities and objectives still need to be communicated in a way that makes sense to operators who are the ultimate sponsors or stakeholders. This means that an innovation program needs to be successful tactically, but it also needs to have a robust corporate strategy that defines, aligns, and eventually supports the core operations. It is possible to translate the innovation objectives into corporate terminology. Those programs which fail to make these translations will eventually struggle to make sense of innovation goals in the context of operations goals. Those programs which make these translations will find it easier to garner and maintain support from corporate leaders despite the ideological differences inherent between the two structures.

Reason #12) The most powerful ideas are incremental (not “disruptive” or “break-through”).

Organizations depend on big ideas but culture depends on little ideas. This point is best made by looking at Minnetonka Public Schools, a K-12 public school district in Minnetonka, Minnesota. The innovation program at MPS is entering its sixth year in 2016. In addition to executive-lead, district-wide innovation initiatives like full-day language immersion, 5th to 12th grade 1-to-1 (a term used for providing a laptop or iPad for each student in a given grade), and other strategic endeavors, the innovation program has given each employee in the district a platform to express their ideas for change and transformation. Treating each employee as a resource for innovation is a trend that has been embraced by a growing membership of corporations but has not largely taken shape within public school districts. This makes the overwhelming success of the MPS innovation program that much more impressive. And most remarkably, it is not the marque innovation that the program values the most. It is the program’s ability to manage a balanced portfolio of large, medium, and small projects (size determined by a combination of effort and investment and not necessarily “incremental” or “break-through” as defined by most innovation thought-leadership). In a letter written by Eric Schneider, Assistant Superintendent for Instruction and the leader of the district’s innovation program, he explains the value of little ideas:

We are in our fifth year of running an enterprise innovation program and the results for our organization have been significant.  We have harvested several game-changing “big” ideas that have moved into the education marketplace and are getting strong return on investment.  Our big ideas become strong intrepreneur opportunities for our employees; we now have a team of innovation project leaders who are all having a rocket-ride experience based on the organization’s support of their innovative ideas.  An important part of our success, however, has been our ability to simultaneously focus on “small” and “medium” ideas in an effort to build trust with our employee group and drive engagement. 

Our belief has been that the big ideas don’t necessarily appeal to the typical front-line employee. They motivate the C-level group to continue investing in the innovation program.  However, without an engaged employee group, we will never get from our staff the raw materials that lead to game-changing ideas. That is why we train our innovation coaches to respond quickly to small ideas, because those are the innovation stories that really spread across the work group and motivate employees to either get involved or stay involved.

So much corporate innovation is focused on the big idea, the game changer, the disruptor, the market creator. As Eric puts it, this makes sense, as the big idea is what engages the C-level and insures investment. But it is making small investments in simple, practical ideas that empowers regular employees that has the long-term benefit. And with greater employee engagement, the odds of finding the next “big” idea increases.

Of all of the trials an innovation program faces in order to be effective, the question undoubtedly becomes, “what should we focus on first?” The simultaneously simple and nearly impossible answer is “all of them.” This is what makes innovation so incredibly difficult within an organization that has not been designed to accommodate it. And yet, by trying to manage all of these priorities at once, by keeping them all conversationally active, then like the three princes of Serendip, serendipity will occur and great riches and fortunes will follow.

The thought-leadership in this article was co-developed by Eric Schneider, Assistant Superintendent of Instruction, Minnetonka Public Schools