Measuring the Value of IT Investments,

make better investment decisions – a key to supporting the President’s Management Agenda. [1]

OMB Exhibit 300 - here’s how VMM can help:

The data we gather during the VMM process can also help us prepare OMB Exhibit 300, Capital Asset Plan and Business Case, which requires a rigorous analysis and justification to support funding requests.

You’ll be able to fill in the Exhibit 300 sections listed below:

ü Summary of spending

ü Project description and justification

ü Performance goals and measures

ü Alternatives analysis

ü Risk inventory and assessment

ü Project and funding plan

VMM data will also support the following sections:

ü Program management

ü Acquisition strategy

ü Enterprise Architecture

ü Security and privacy

ü GPEA

Before we discover the details of the business case, let’s first learn more about what VMM is and how it can help us, and the basics of Return on Investment.

2. An Innovative Approach to Measuring Value: the Value Measuring Methodology (VMM)

The Federal Chief Information Officer’s Council, Best Practices Committee has recently released a report, Value Measuring Methodology (VMM), How-To-Guide, [2] which focuses on sound investment management. VMM is a proven toolkit of existing techniques used to define, capture and measure both quantitative

and qualitative value associated with information technology investments.

The four steps involved in the Value Measuring Methodology are summarized below:

1. Develop a Decision Framework. This is the foundation for designing, analyzing and selecting a project to invest in, and then managing that investment. The framework consists of value (benefits), cost and risk structures.

The value structure describes and prioritizes benefits, and identifies and prioritizes measures of success. The structure consists of five perspectives on value:

Customers (users) – providing services electronically;

Social – benefits to society as a whole;

Government/Operational Foundational – improvements in Government operations and enabling future initiatives;

Strategic/Political – contributions that will help achieve strategic goals, priorities and mandates; and

Government Financial – financial benefits to both sponsoring and other agencies.

We’ll come back to these important value factors later.

A cost structure is developed to ensure a complete and comprehensive estimate. (Incomplete or inaccurate estimates can result in exceeding our budget.)

A risk structure is developed to identify potential risks that can jeopardize the project. Plans to mitigate those risks are also developed. Risk tolerance – what’s the maximum performance slippage you will accept, etc., - is included here.

2. Perform an alternatives analysis. Our value, cost and risk structures will be used here to compare the new investment against alternatives. We’ll also want to conduct uncertainty, sensitivity and risk analyses to better calculate cost and value.

3. Gather our cost estimates, value projections and analyses; and develop value and risk “scores”, along with decision metrics that compare value to investment and financial ROI.

4. Communicate and document the findings. Budget justifications, including OMB Exhibit 300, can now be prepared; and the lessons learned can be used to make improvements.

3. Return on Investment

Knowing how to compute Return on Investment (ROI) will help us determine the financial benefits of our investment.

Simply stated, ROI is the profit or loss resulting from an investment transaction, usually expressed as an annual percentage return.

Simple ROI = (gains – investment costs) ¸ investment costs x 100.

For example, if a t-shirt manufacturer spends $2 to manufacture one t-shirt and sells 50,000 t-shirts at a price of $10 each, the costs would = $100,000 and the gains would = $500,000. Using the formula above, ROI would be calculated as follows:

ROI = $500,000 - $100,000 = $400,000;

$400,000 ¸ $100,000 = 4;

4 x 100 = 400%.

This calculation works well in situations where benefits and costs are easily known.

However, investments frequently involve financial consequences that extend over several years. In this case, the metric has meaning only when the time period is clearly stated. Net Present Value (NPV) recognizes the time value of money by discounting costs and benefits over a period of time, and focuses either on the impact on cash flow rather than net profit, or on savings.

A meaningful NPV requires sound estimates of the costs and benefits and the use of the appropriate discount rate.

Microsoft Excel can automatically calculate NPV for you!

Once the NPV has been determined, the formula now becomes:

ROI = NPV of savings ¸ investment costs x 100. An investment costing $1M that has an NPV of savings which = $1.5M would have an ROI of 150%.

ROI = $1,500,000 ¸ $1,000,000 = 1.5;

1.5 x 100 = 150%.

A two-page document entitled, The Value of IT Investments: It’s Not Just Return on Investment, provides more detail on Return on Investment and Net Present Value. This document can be found at http://www.cio.gov under the Best Practices section.

4. A Real World Example of Finding Value, Even With a Low ROI: the XML.gov Registry/Repository Business Case

When comparing alternatives to invest in, our project may turn out to have a low ROI. But, as we will see in the following example of a real world business case, having a low ROI may be justifiable.

When information is shared electronically, it must not only be exchanged, it must be manipulated and interpreted. The eXtensible Markup Language (XML) enables the sharing of electronic information because it provides a flexible, non-proprietary way to annotate or ‘tag’ information so that it can be exchanged and interpreted by dissimilar computer systems.

The XML.gov Registry/Repository business case addressed the need for coordination and interoperability of XML registry/

repositories sponsored by Federal agencies. It considered whether or not the Government should invest in a common XML registry/repository to unify and interoperate with registries operated by different Federal agencies. It also considered whether or not XML.gov should serve as a central portal linking to all Federal

agency registry/repositories and provide a central publishing forum for agencies that elected not to build their own registry.

A registry is defined as an electronic listing of specifications (schemas, document type definitions, etc.) and their locations.

In the current Federal environment, XML activities and registries are disjointed, not interoperable, and there is limited awareness of their content and potential applicability to Federal IT initiatives. Also, the technological specifications for these registries vary from registry to registry. An investment in a Federal registry/repository would not only provide a central portal to retrieve artifacts in registries across the Federal Government, it would also enable Federal agencies without their own stand-alone registries to publish artifacts.

The XML.gov Registry/Repository initiative will provide a consolidated governmentwide approach to XML definition, storage

and reuse among all Federal agencies and their partners. By providing a means of electronic information sharing and enabling E-Government transactions, the initiative will yield efficiencies and directly benefit customers and stakeholders, including Federal, State and local government entities, private industry, the scientific community and the public.

Value Factors, Benefits and Assigning Weights.

The XML.gov Registry/Repository Business Case was prepared using the Value Measuring Methodology (VMM). This methodology is compliant with guidance from the Office of Management and Budget (OMB) and incorporates both public and private sector analytical best practices.

Let’s take a closer look:

The expected total investment costs of a federated model of government-sponsored registries linked and governed by a primary

registry/repository – XML.gov – is $7.7M, with a total life-cycle cost of $59.4M. Costs include establishing and operating a primary Federal registry/repository that will link all individual registries, costs of individual agency registries that are assumed to exist, and

costs associated with new registries that will be created in 2004 – 2013.

The alternatives compared were:

1) the base case – maintaining current (disjointed and not interoperable) and future registries

2) the federated registry/repository; and

3) the centralized registry/repository - all existing Federal XML registries would effectively be shut down and XML artifacts migrated into a centralized registry.

Using VMM, the cost of the investment was captured and quantified, along with the full range of value it provides to direct users, stakeholders and the Government itself. The five value factors and XML.gov Registry/Repository initiative-specific measures were identified and prioritized as follows:

Table 1: Value/Benefit Weights

Each alternative is analyzed against the same decision framework – comparing apples to apples – in which all value is translated onto a single scale. The completeness of this approach provides decision makers with the information necessary to explore, understand and make decisions based on the relationships between value, cost and risk. (Later, we’ll learn how each alternative stacked up.)

What about the ROI?

Cost savings and avoidance were compared to the cost of investment to determine ROI.

The ROI for the base case was 0% because the Government would receive no financial benefits by maintaining the status quo.

The ROI for the federated model turned out to be 544%.

The third alternative – the centralized model – had an ROI of 1,343%, due to substantial cost avoidance. There would be no independent XML activity and registry proliferation in the Federal space, including associated costs of operations and maintenance.

So, with a much lower ROI than the centralized model, why would the federated model be the preferred alternative?

How the federated model performs.

Will this alternative “perform” based on the standards for the E-Government Project Portfolios, which are described in Implementing the President’s Management for E-Government, E-Government Strategy?

E-Government projects are organized into portfolios, defined by the customer segment they serve: [3]

Government to Citizen (G2C). The objective of the G2C portfolio is to provide one-stop, on-line access to information and services to individuals.

The XML.gov Registry/Repository will support the 24 governmentwide E-Government initiatives [4] by facilitating

standardized XML-based data that can be leveraged by all Government web sites in a consistent manner.

Government to Business (G2B). The goals of the G2B portfolio are to reduce burdens on business, provide one-stop access to information and enable digital communication using the language of e-business (XML).

Businesses will have reduced reporting and regulatory burdens through the electronic searching of Federal regulations and the electronic filing of reports and inquiry responses. The data standards to support these electronic processes will be XML-based and housed in the Federal registry/repository and accessible through individual agency registries, as well as the primary registry.

Government to Government (G2G). The G2G’s primary goal is to enable Federal, State and local governments to more easily work together to better serve citizens within key lines of business.

Governments will be able to create seamless processes by linking systems and standardizing data formats through the use of XML schemas, which will be housed in the Federal registry/repository.

Internal Efficiency and Effectiveness (IEE). The Federal Government must modernize internal processes to reduce costs for Federal Government agency administration. The IEE portfolio’s focus is to apply industry best practices to improve Government operations.

Common XML artifacts will foster reuse among Federal developers and integrators and create a vehicle for standardized information exchange within and between agencies. Also, XML artifacts will be used as core “building blocks” to institute industry best practices (supply chain management, knowledge management, etc.).

Because the XML.gov Registry/Repository is a foundational, governmentwide capability, it successfully satisfies the performance objectives of all four portfolios.

Just as important, though, are the five perspectives on value. Here’s how all three alternatives – the federated model first, then the base case and the centralized model – stacked up:

Customers/Users – The federated model delivers the most value to direct users by maximizing the ease of use of XML resources and information, and by providing a single authoritative source for definitions of XML artifacts for government-unique requirements.

Social - The federated model scored very well in this area due to coordination and streamlining of intergovernmental data collection sharing and more efficient use of taxpayer dollars.

Government/Operational Foundational – The federated model is by design a foundational effort to improve the efficiency and effectiveness of “back office” communications and transactions. (Value scores were highest for this factor.) It delivers the most foundational value by enhancing the IT performance of Federal agencies, increasing productivity and efficiency in Government operations, facilitating information-sharing among disparate systems and agencies, and improving Federal interagency collaboration.

Strategic/Political – The federated model also scored very well in the strategic/political area because it fulfills the strategic goals of the President’s E-Government agenda and facilitates the building block for homeland security coordination.

Government Financial – The federated model received a low score in this area due to the fact that a primary Federal registry would be established and operated in addition to and in tandem with individual registries established by different agencies across

the Government with which it is linked. All of the alternatives in this business case reflect total costs to the Government as a whole. The costs of the federated model include those associated with

individual agency registries that were projected to proliferate over the lifecycle of the XML.gov Registry/Repository initiative.

If we refer to the chart on page 10, where value factors and benefits have been weighted, we see that benefits for the Direct User value (28%), the Government Foundational/Operational value (25%), and the Strategic/Political value (20%) are the most important factors. Individual benefits with the highest scores were:

o Ease of use of federated registry/repositories for direct users;

o Operational efficiencies due to the interoperability and coordination of government-sponsored registries; and

o Improved coordination, communications, and knowledge and data sharing between Government entities facilitated by interoperable Federal registries with standardized artifacts.

The XML.gov Registry/Repository will provide high value in these key benefit areas. It had the highest total value score of all three alternatives and was, therefore, selected as the preferred alternative.

The base case was not selected because this alternative is by its very nature in direct opposition to achieving the fundamental benefits associated with the use of XML technologies, specifically, efficiency benefits that arise out of the standardization of artifacts; the interoperability of registry/repositories and the reuse of artifacts to standardize communications and transactions between business partners. No consolidation or coordination of registries takes place and no associated cost savings are realized.

The centralized model received a high score for its Strategic/ Political value (a single, custom-built registry would be much easier to align to the Federal Enterprise Architecture Business Reference Model [5] and it facilitates the building blocks for homeland security coordination) and for its Government Financial value (cost avoidance due to the fact that no federally-sponsored registry/repositories would exist outside of the centralized registry).

However, this alternative scored poorly for its Direct User value and had mixed results for the Government Operational/ Foundational value and the Social value, where it scored very

poorly for its efficient use of tax dollars. Even with the higher scores in some areas, its total score was lower than that of the federated model.

The following table summarizes scores for all three models:

Table 2: ROI and Total Value Score Comparisons

Summary

Base Case

Federated Model

Centralized

Model

Total Cost

125.7

59.4

23.1

Investment Cost

19.3

7.7

4.9

ROI (Cost Savings and Avoidance to Investment

.00%

543.84%

1,342.86%

Total Value Score

22.57

58.48

49.75

Sometimes it’s OK to have a low ROI.

It is essential to consider total value, not just pure ROI metrics.

Even though the federated model did not receive the highest ROI number, it will provide high value in key benefit areas.

The federated model scored very high in the Direct User, Government Foundation/Operational, Strategic/Political and Social value factors. These value factors carry weights of 28%, 25%, 20% and 8%, respectively, of the total value. The Government Financial value factor carries a weight of 19%. Even though the federated model’s ROI was lower than that of the centralized

model, when all value is totaled, the federated model is the clear winner. It provides a structure to realize the full efficiency benefits

associated with XML, therefore, having a significant impact on the value provided to all users of XML technologies and to E-Government stakeholders and leaders.

Return on Investment (ROI) is one metric that is very useful. However, it’s internally focused and gives a dollar value only. It’s essential to perform a rigorous analysis of all value factors. As we have seen in the XML.gov Registry/Repository Business Case, the benefits of a federated XML registry far outweigh the investment costs.

5. Summary

Let’s look at how the federated model – XML.gov - links to the President’s Management Agenda. [6]

The President’s vision for Government reform is guided by three principles. Government should be:

o Citizen-centered, not bureaucracy-centered;

o Results-oriented; and

o Market-based, actively promoting rather than stifling innovation through competition.

The federated model satisfies all three principles:

It is citizen-centered. The XML.gov Registry/Repository will support the 24 governmentwide E-Government initiatives by facilitating standardized XML-based data that can be leveraged by all Government web sites in a consistent manner. Further defining of XML components will lead to additional data being made available in formats that are easily accessed and displayed by current web sites.

Businesses will have reduced reporting and regulatory burdens through the electronic searching of Federal regulations and the electronic filing of reports and inquiry responses. The data standards will be XML-based and accessible through individual agency registries, as well as the primary registry.

Governments will be able to create seamless processes by linking systems and standardizing data formats through the use of XML schemas, which will be housed in the primary registry.

It is results-oriented and market-based. Common XML artifacts will foster reuse among Federal developers and integrators, creating a vehicle for standardized information exchange within and between agencies. This will improve cost effectiveness for the Government.

XML artifacts will be used as one of the core building blocks to institute industry best practices – data can be tagged and described in XML, and schemas and components can be shared via the primary registry, a common practice in industry.

To summarize, this White Paper has presented a business case that used a new methodology for analyzing E-Government investments. VMM is a thorough and rigorous analytical approach to evaluating and managing investments. It considers the perspectives of all of our stakeholders. By developing a decision framework, and the value, cost and risk structures, we’ll know the importance of each value factor and be able to compare “apples to apples.” The measures we identify become our standard management “yardstick” against which to judge investments. These same measures will help us throughout the life cycle of the investment to ensure our project is “on track.”

Our ROI is very important, but it’s not the only important value factor. As we have seen in the XML.gov business case, the ROI was a positive number, as required by OMB Circular A-11 [7], but was lower than that of another alternative. But the benefits to be derived were important enough to make XML.gov the preferred alternative.

[1] The President’s Management Agenda, Fiscal Year 2002, Executive Office of the President, Office of Management and Budget; http://www.whitehouse.gov/omb/budget/fy2002/mgmt.pdf

[2] Value Measuring Methodology, How-To-Guide, CIO Council, Best Practices Committee; http://www.cio.gov/ under Best Practices

[3] Implementing the President’s Management Agenda for E-Government, E-Government Strategy, Executive Office of the President, April 2003, p. 9; http://www.whitehouse.gov/omb/egov/2003egov_strat.pdf

[4] Implementing the President’s Management Agenda for E-Government, E-Government Strategy, Executive Office of the President, April 2003, pp. 24-37; http://www.whitehouse.gov/omb/egov/2003egov_strat.pdf

[5] Federal Enterprise Architecture Program Management Office, Business Reference Model; http://www.feapmo.gov/feabrm.htm

[6] The President’s Management Agenda, Fiscal Year 2002, Executive Office of the President, Office of Management and Budget, p. 4; http://www.whitehouse.gov/omb/budget/fy200

Table of Contents

1. Introduction

2. An Innovative Approach to Measuring Value: the Value Measuring Methodology (VMM)

3. Return on Investment