Be Careful in the Custom Infrastructure Wasteland

-->

An interesting article was posted today at CIO Zone. The article was titled “Cray Teams with Microsoft on Cloud Computing.” Apparently Microsoft and Cray plan to work together on cloud computing systems. Cray is a supercomputer company. Microsoft is a well known provider of software for PC’s and servers... I find it interesting in that on first blush these are strange bedfellows and neither is “top of mind” in this market. It appears to me that they’re trying to get into the game.

The two have formed a partnership whereby Cray’s custom engineering group will work with Microsoft Research (italics are mine) to explore and prototype cloud computing infrastructures. They’re focus is to “dramatically lower the total cost of ownership for cloud computing data centers” through the application of “latest breakthroughs in high-density packaging and cooling technologies.” According to Cray this is the engineering group’s first move into the commercial market. Until now they have designed and delivered custom solutions to the specifications of an individual customer.

In previous posts I have referenced the Everett Rogers’ Diffusion of Innovations and Geoffrey Moore’s application of this idea to the marketing of technology solutions in Crossing the Chasm. In those models today’s announcement involves two innovators who are enthusiastic about their technical prowess and the contributions they can offer. They will likely construct some impressive prototypes and go to beta with a few customers (including themselves). Then they will face the challenge of getting material adoption from the market at large.

If history serves the Cray and Microsoft technology enthusiasts and visionaries will have a hard time convincing the wider “pragmatic market” to adopt their solutions. For one thing, as enthusiasts and visionaries they will tend to have a tin-ear to the non-technical buying patterns of the market at large. The pragmatic market wants providers with track records that cover not just the technical but the financial and support aspects of the buying decision as well. They want providers with organizational infrastructure and history. They want reliable, long term partners from that space. This is a new market for the partners.

For another thing, by the time the partnership is ready to go the relative importance of their packaging and cooling technologies will likely diminish. Theirs may always be better than what competitors offer but it will be hard to be so much better that it sways buying decisions in the face of other benefits competitors will bring. In large scale computing like Das Kloud everyone is working on energy conservation. So-called Green Computing makes everyone feel good and saves a lot of money. Other technically advanced and well heeled, long-time players are hard at work on their own efficiencies. Cray’s advantage will have to be substantive to drive very many decisions by customers.

Still, some IT management can’t resist the temptation to use “the new new thing.” In one ancient example a colleague was surprised to find that the IT director, unsatisfied with the IBM operating system that came with his mid-range device, had written his own, “Chief/370.” In another, a major insurance company’s technical services group was unsatisfied with the transfer rates and job control capabilities in the standard labels for data center tapes and wrote their own. They were internally efficient and had no upward compatibility at all and could not share files easily with others.

More recently companies are building their own content management systems or buying “unique and innovative” Enterprise Service Busses. In all of these cases the maintenance and support of the technology becomes an internal, un-leveraged expense versus a shared expense across all users of the technology. Sometimes you don’t even get to an operating solution. Not long ago a major financial services firm planned to rewrite all of their 30 year old core account management systems in a solution so elegant it required proprietary, early-stage processors. Soon enough the runaway project collapsed and hundreds of millions of dollars went wasted. The hardware vendor collapsed too.

Working on Step 2

Previous posts have shown the total cost of ownership relationship between developing systems and the down-stream costs of maintenance, support and operation. If the underlying infrastructure you are using is unique to you or shared by a very few others your total cost of ownership for that portion of your applications portfolio will be high, in some cases prohibitively.

On the other hand, if the only thing you do is employ technologies that are widely installed and thoroughly market tested it is unlikely that you will be able to generate any material competitive advantage from information technology. For most companies it becomes important to not only keep an ear to innovations but to pursue them from time-to-time. How to proceed?

My first suggestion is that you soberly set your context in business terms. Either you work for an aggressive company or you don’t. There is an expectation that managers try new things in some companies. In others managers have less leeway. All companies experience change. Perhaps your company is experiencing more change in the nature of the competitors you are facing, in how people are winning (or losing), in what the future holds. The more aggressive your corporate personality, the more leeway managers have, the more change your industry is facing the more innovative risks you should consider.

Still, when considering risks be hard nosed about it. Malcolm Gladwell published an instructive article in The New Yorker in January describing how true entrepreneurs like Ted Turner, Sam Walton and the hedge fund manager John Paulson, were not actually wild risk takers. Instead, they were thorough, detail-oriented and, once they fully understood an advantage, very aggressive. The people who thought they were risk takers, or crazy, hadn’t done the homework that showed they had a sure thing on their hands.

You may well want to use a burgeoning rules technology or a better business intelligence platform or cloud computing from Cray and Microsoft. Make a thorough analysis of all your options. Know the total cost numbers objectively, deeply and coldly. Do not sugar-coat your internal support costs or the risk that you’ll have to provide more support than the vendor or your internal enthusiasts think. It’s always better to have others in the buying pool with you. Being the only owner of Chief/370 is a disaster. Have some company you can talk openly and bluntly with.

Lastly, give yourself an out. When sitting in design meetings you’ll have discussions about establishing structured interfaces between modules and systems or using an SDK from the vendor (that may well be under design as you’re talking). These are important discussions. A friend often said that “an interface is a freedom creating device. If you obey it you can do whatever you want on your side.” In this sense establish an interface between all other systems and the technology innovation. If the innovator’s solution doesn’t prove out you can move on to another solution. By then the market may well have surpassed the innovator. Unless you like failure do not accept the compromise of speed-to-market for fudging on this out. If the project gets behind or gets costly then adjust the budget and timeline or get out. The minute you customize the solution it’s broken and it’s yours and that’s rarely a good thing.

Let's Get Small

(this post is expanded, in content and images from an article published at IT Business Edge)
After toiling in obscurity, learning the hard way about stand-up comedy and how to play the banjo, Steve Martin started catching on in the 1970’s hitting the big time with a comedy album titled “Let’s Get Small.”




The title and the bit by that name are nominally an absurd twist on the drug culture of the time and the phrase “Let’s get high.” As it turns out there is a theory that Martin was actually prescribing how to manage a portfolio of IT projects. Researches have debunked the theory that Martin was recommending that IT management parade around the office with arrows through their heads and wearing Groucho Marx glasses…

The longer each project is the less likely that you can properly define all its goals much less achieve them much less ensure yourself and your colleagues that the effort was worth it. If you have a long-term goal, a roadmap, that’s fine. Get there in pieces with tangible benefits coming from each step.

This reason is that, no matter what we’re doing, from target sports to planning out a systems initiative, our aim is much better on short shots than it is on long ones. The longer the shot, or project duration, the harder it is to see, understand, explain and focus on the target. Even if you can do those, the longer the project duration, or shot, the harder it is to actually take aim at all much less deliver what you intended.

With a little thinking you can predict how far off your aim is likely to be and, using that information, have a sensible discussion about what should be the limits on individual project duration. To keep the example simple let’s assume that whatever you think is the right thing to do today will, in one year’s time, be off by 20%. There are many reasons, among them: technology will change, vendors will collapse, the economics of your business will change, skill sets will enhance, and new competitors will enter your market forcing new ways to compete.

If you had $3 million to spend and you embark on a large-scale 3 year project, due to compounding year-over-year change your vision will be off by 20% after one year, 36% after two years, and nearly 50% after 3. Few projects are managed that way except for the “we have to replace the core systems” projects. The compound inaccuracy of vision and reality is one of the reasons so many of them result in runaway projects (I’m thinking of three that I’m familiar with right now).

If you took that same $3 million and spent it on a series of $1 million dollar, 1 year projects the visions would miss the targets by 20% each year, but never more than that. Break it down again into 6 month chunks and the maximum mis-alignment is 10%. Go “extreme” and require all projects to deliver value in 3 month chunks and you’re never off by more than 5%:




In terms of protecting the bottom line there is no question that making projects small is the best possible thing to do, to a point. Just going from one-year projects to maximum 6 month projects enhances the value for the dollar by 10%. This is big money especially in tight money times.

This analysis must be tempered by the time, effort, and distraction you receive from repeatedly going back to the drawing board to set new goals. If you chunk up a long-term project one frequently runs the risk of losing funding from phase to phase as “higher priorities” arise or as the ROI on Step 3 is attacked as too weak to fund even though it’s fundamental to Steps 3-6. Then there’s the ego/glory factor. Business and IT managers alike often prefer the big budget projects with the big resume impact they carry.




This is where a plan or a roadmap comes in. Susan Cramm suggests you get the money but spend it right. What she’s on about here is something akin to the Apollo program. There was a goal set out, get to the moon by 12/31/69, but the plan in 1961 only laid out the macro steps and the near-term efforts. On an effort like that, since it had never been done before, they couldn’t plan “beyond their headlights.” Most business projects are not that sophisticated. That said, the underlying business conditions and assumptions are constantly changing and so must project definitions.

Speaking of cramming… Do NOT use this as a license to request 7 months of deliverables and their required effort in 4 months time. This is an exercise in maintaining “alignment.” There are myriad ways people try and take time off a professional job. My personal favorite is “don’t do testing,” which is fine if you want to be highly productive at delivering systems that don’t work. The value of this approach lies only is keeping resources and time for tasks as they must be but re-adjusting your aim on a regular and timely basis.

If you set and execute a long term vision, planned out up front without down-stream adjustment you will miss the mark by a wide margin. Set long term goals but limit your systems definitions, designs and efforts to short-as-possible projects with tangible benefits. At the end of each take a short breath, reset your sights, and do it again. Get small. It’s the way to win.

Raising the ROI of IT

IBM’s Institute for Business Value recently published a report titled “The New Voice of the CIO.” It is based on an exhaustive series of interviews with CIO’s across industries and around the globe. Based on insights from CIO’s the study attempts to describe what makes a CIO, and by extension, an IT organization successful (for more background on the study’s findings try this link). The chart below is the centerpiece of the study and this blog’s focus is on the concept at its center, raising the ROI of IT. IBM finds that in order to raise the ROI of IT a CIO and the IT team must be “savvy value creators” while cutting costs. One CIO quoted in the study says “the balance between new projects and cost control is the dichotomy of my life.”

Source: “The New Voice of the CIO” - IBM Institute for Business Value

Creating value combines selecting when and where to invest and when doing so, hitting sweet spots. In terms of “value” the CIO’s IBM spoke to are looking to enable superior customer experiences, help the business cope with burgeoning amounts of information, and to extract value from that data especially to understand what customers want. In the near term this includes collaborative relationships with customers, transparency, and active input from and interaction with customers.

Cutting costs involves increasing either or both efficiency and effectiveness. One CIO in IBM’s study says that companies “need to position IT to handle increased activity with minimal additional cost.” This is creating leverage from efficiency. Throughout the history of IT in business, three tools: standardization, automation and centralization have proven to be well worn tools for generating efficiency. From the original one’s and zero’s to Assembler and Cobol, from a dogs breakfast of machines and operating systems to System 360/370 and Dec VAX and the like, from various PC OS’s and word processors to MS-DOS and Word, economies of scale and standardization has provided efficiency (though some argue about the cost of lost effectiveness or economies of scope).

Standardized business processes for IT are also a key method to be efficient and effective. ITIL, COBit, CMM and similar approaches create a lexicon and a definition of what best practices are. They enable measurement, training, review and audit for continuous improvement. They help limit home-grown innovation to those areas of IT management that particularly need it. But they can be overused and over applied. CMM Level 5 is almost certainly overkill as a general standard although for certain business units at certain times it may be a requirement. ITIL 3.0 won’t save you costs if your needs are pieces of 2.0, a slice of 3.0 in one or two key areas, and leaving the rest alone. Targeting the scope and sophistication of an applied standard says much about efficiency and effectiveness.

Here is, admittedly, a moderately academic question. Why must it be so that raising the ROI of IT requires both value creation and cost cutting at the same time? Can’t one keep efficiency constant in return for continuous value creation? For a short time, perhaps, but long term the math won’t let you.

Working on Step 2

The reason why enhancing the ROI of IT mandates both value creation and cost cutting is that each new item of value built today requires both operations and support resources and maintenance and enhancement resources for the useful life of the system, application or feature that’s been built.

There are in today’s world myriad options for how systems are developed, the environments in which they are run and the methods used to keep the systems current and to enhance or adjust them for changing business opportunities. Because of this “your mileage may vary” but the dynamic of needing to reduce the cost of owning systems in order to create budget to develop new ones won’t. Left unchanged in efficiency and effectiveness and IT budget is starved of funds for innovation.

If you change the efficiency by which you maintain and enhance systems over time you can have significant impact on IT spending over time. The traditional assumption is that for every 5 person team building a system one person must be allocated to maintain that system over time. In other words, each Development dollar, on average, results in 20 cents of Maintenance and Enhancement (M+E) dollars. D --> M+E = 20%.

Some organizations claim 15% or lower impact. On the other hand, Bart Perkins of Leverage Partners points out that if you buy a package that is a mis-fit in function or scale the M+E load can be significant. He cites a company with a small IT team and a small IT budget that implemented PeopleSoft when a more middle-market solution may have been advisable. The result is a 40% D --> M+E ratio for that firm. Similarly a large company could buy a middle-market solution and enhance it themselves, when they should have bought PeopleSoft or its equivalent, with a like result. Fit is important.

In the initial versions of this analysis when “IT Operations” meant data centers, the ratio of Development dollars to Production dollars was one half. D --> P = 50%. Anyone who has priced out the cost of a secure data center knows that prices can be all over the map.

In one situation I observed a customer in the financial services market had such stringent requirements for back-up, recovery, fail-over and the like that the projected cost of operations for a major new system doubled over the configuration normally seen by its vendor. Policies have an impact. In another example, one client’s charge-back policies were not regularly modified leading them to under-charge for services that required relatively high manual activities and to over-charge for once leading-edge technologies that were now highly reliable and required little IT Operations personnel to operate. Their costs for like applications portfolios were double their peers’.

If you assume $1,000 spend on development each year and D --> M + E ratios and D --> P ratios of 20%, 30% and 40% in subsequent years you get quite a spread. The development spending is the same but the total IT spend over the term is 20% higher or lower based on lower or higher efficiency by which systems are operated, maintained or enhanced.

If you project into the endless future, especially with moderate to low efficiency assumptions, the IT spend quickly outpaces any prediction you may have for increases in overall company revenue. “Straight line” a model like this and soon all spending is IT spending and that’s a problem. Relying on Moore’s Law to reduce the cost of technology is not enough. This is why IBM found what they found. Unless the IT organization constantly pays attention to efficiencies their budget will be consumed by Maintenance and Operations. Development requests will go un-served and IT management will soon be replaced by people who can get something done.

As the man said, “the balance between new projects and cost control is the dichotomy of my life.”