5 FAM 670
INFORMATION TECHNOLOGY (IT) PERFORMANCE MEASURES for
project management
(CT:IM-258; 12-04-2018)
(Office of Origin: IRM/BMP/GRP)
5 FAM 671 WHAT ARE IT PERFORMANCE
MEASURES and why are they required?
(CT:IM-248; 12-04-2018)
a. Performance measures are benchmarks for evaluating
how IT investments can be more efficient, effective, and obtain better results
to support an organizations mission, goals, and objectives. Measurement is
vital in determining what a project is accomplishing and whether the results
are being achieved.
b. The Government Performance and Results Modernization Act of
2010 requires the Department to establish performance plans with
performance goals. These goals should define the level of performance to be
achieved by a program activity and the need to express these goals in an
objective, quantifiable, and measurable form. Office of Management and Budget
(OMB) Circular A-130 mandates agencies use performance measures to ensure that
IT investments are managed efficiently and effectively.
c. IT metrics must be aligned with the Departments
strategic goals and mission as outlined in the Joint State/USAID Strategic Plan
and the performance criteria specified in the Joint Performance Plan (see 5 FAM 130).
d. Department of State entities must establish
performance measures that monitor and compare actual performance to planned
results as stated in the Department of State IT Strategic Plan (see 5 FAM 130).
5 FAM 672 scope of this policy
(CT:IM-74; 05-03-2006)
a. This subchapter applies to program and project
managers in all Department of State entities both domestic and abroad, and is
the official policy for measuring IT performance. It also introduces the
Performance Reference Model (PRM) mandated by OMB.
b. This policy is designed to enhance managers
understanding of information technology performance measures (see 5 FAM 678.4).
5 FAM 673 AUTHORITIES FOR THIS POLICY
(CT:IM-74; 05-03-2006)
The authorities for this policy are covered in 5 FAM 113.
5 FAM 674 applicable definitions
(CT:IM-74; 05-03-2006)
Baseline A starting point or
initial value for a program or project measurement indicator.
Business Case A clear and
concise justification describing what the program or project entails, why the
product or service is necessary, how much it will cost, what risks are
involved, how work and progress will be tracked and measured, and the timeframe
for completion.
Business Reference Model (BRM) A
framework that facilitates a functional (rather than organizational) view of
the federal governments lines of business, including its internal operations
and its services for citizens.
Capital Planning and Investment Control
(CPIC) A decision-making process, directed by the Departments
E-Government Program Board (E-GovPB), for ensuring that information technology
investments integrate strategic planning, budgeting, procurement, and the
management of IT in support of the Departments mission and business needs.
Customer Results Measurement Area
The part of the PRM that captures how well an enterprise-wide or specific process
within the Department is serving its customers, and ultimately citizens.
Electronic Government Program Board
(E-GovPB) An advisory entity to the Under Secretary for Management
that addresses the full range of Department E-Government and IT investment
portfolio and project management activities.
Electronic Government Program Management
Office (E-Gov PMO) The office that ensures the completion of all
program elements related to the Departments IT investments for meeting
E-Government guidance and ensure that major milestones are met throughout all
stages of the CPIC process.
Enterprise Architecture The
practice of applying a comprehensive and rigorous method for describing a
current or future structure for an organizations processes, data, information
systems, technology, and organizational sub-units, so that they align with the
organizations core goals and strategic direction. Although often associated
strictly with information technology, it relates more broadly to the practice
of business optimization in that it addresses business architecture,
performance management, and process architecture as well.
Enterprise Architecture Line of Sight
Diagram A diagram for each IT initiative that brings together all of
the components of the Federal Enterprise Architecture Models (e.g., BRM, PRM,
etc.) onto a single graphic illustration that reflects how value is created as
inputs (e.g., technology) are used to create outputs via the business processes
and activities, which in turn, impact outcomes (i.e., mission, business and
customer results).
Goals A concise and measurable
description of one or more related actions that are necessary to attain a
tangible milestone toward the achievement of an objective.
Human Capital Measurement Area
An area of the PRM that captures the human capital aspects of performance.
Inputs The type and amount of
resources that are required and used to deliver a government service.
Information Technology Performance
Measurement The indication of what a program or project is
accomplishing and whether results are being achieved while supporting business
processes and strategic program and project outcomes.
Information Technology Investment
A capital IT asset acquired through the acquisition process that must be
managed throughout its life-cycle.
Lines of Business Descriptions
of the services and products the Government provides to its stakeholders.
Line of Sight A concept that
clearly articulates the cause-and-effect relationship between inputs, outputs,
and outcomes.
Measurement Areas Performance
areas that identify the highest level of the Federal Enterprise Architecture
(EA) Performance Reference Model (PRM) framework and capture aspects of
performance at the input, output, and outcomes levels.
Measurement Categories Groups
within each measurement area of the PRM that describe at a high level an
attribute or characteristic to be measured.
Measurement Grouping Subgroups
within each measurement category of the PRM that describe specific types of
measurement indicators.
Measurement Indicators Specific
measures developed by a program sponsor, program, or project manager used to
track, assess, and report the success or failure for a specific BRM line of
business or subfunction, Department, program, or project initiative.
Mission and Business Results Measurement
Area A measurement area within the PRM and major component of the
line of sight that illustrates the outputs or desired results the Department
seeks to achieve.
Objective A description of a
strategic position to be attained or a purpose to be achieved that is tangible
and attainable.
OMB Exhibit 300 A comprehensive
questionnaire designed for major IT investments that is required annually by
OMB to justify funding and summarize the investments business case, describes
what the program or project entails, why the product or service is necessary, how
much it will cost, what risks are involved, how work and progress will be
tracked and measured, and the timeframe for completion.
Operationalized Performance Indicators
Clearly defined specific, measurable, attainable, realistic, and
time-associated metrics used to track progress and determine the effectiveness
and efficiency of IT initiatives.
Other Fixed Assets Measurement Area
An area of the PRM that captures the performance of other assets such as
vehicle fleets, facilities, and other equipment.
Outcomes The end result of
government action, aggregated across multiple agencies and levels of
government.
Outputs The amount of work
accomplished; discrete products, services, or information that pass from the IT
program to the customer (business unit or citizen).
Performance Indicators
Characteristics or properties of resources, processes, customers, or desired
results that the Department uses to measure progress toward projected annual
performance goals and strategic objectives and goals. Indicators are drawn
primarily from bureaus and mission performance plans but may come from
day-to-day operations.
Performance Metrics A set of
standard measures used to identify and evaluate how well specified
characteristics or properties of resources, processes, customers, or desired
results change over time when compared against a baseline value.
Performance Reference Model (PRM)
A standardized measurement framework designed to measure the performance of
IT investments and their contribution to program performance.
Processes and Activities Measurement Area
The area that creates the outputs directly resulting from the process
an IT initiative supports.
Program A group of planned
projects having a common goal, objective, or mission with a defined budget,
management structure, dedicated resources, and does not have a specified start
and end date.
Project A planned task or
undertaking that has been scheduled to achieve a desired result within defined
budget and time constraints; a planned effort with a performance goal,
scheduled start and end dates.
Technology Measurement Area The
area that captures key elements of performance directly relating to the IT
initiative.
5 FAM 675 what are the roles and
responsibilities for measuring IT performance?
(CT:IM-74; 05-03-2006)
The following roles and responsibilities are vital to
ensuring that IT performance measures are developed and implemented
domestically and abroad to ensure that the necessary metrics and performance
indicators are appropriate for accomplishing the stated goals and objectives.
Other responsible organizations are in 5 FAM 115.
5 FAM 675.1 Program Sponsors of IT
Initiatives
(CT:IM-74; 05-03-2006)
Program sponsors within a program or administrative area for
an IT initiative, translate process and business goals into system requirements
and coordinate their implementation. They are responsible for linking
organization and IT performance. The Project Sponsor should be a senior
individual with requisite management, technical, and business skills to lead
the investment or supervise a designated Project Manager. Commitment by the
Project Sponsor to the Departments CPIC process represents accountability for
the investment.
5 FAM 675.2 Business Owner
(CT:IM-74; 05-03-2006)
The Business Owner manages the development, modification,
and enhancement of a program for a specific business area. The business owner,
who may also be the process owner, identifies and defines the program
objectives and goals, process objectives and goals, the business outcomes, and
results to be achieved. The business owner is responsible for identifying,
defining, and specifying the organizational performance metrics. The business
owner is also responsible for capturing or measuring some program performance
data, consolidating all performance data captured and measured by program and
project, and then evaluating the results of the overall programs performance.
Evaluation of the program performance data may require the business owner to
make adjustments to the program plan, schedule, activities, or level of
resources planned for and utilized. Changes made by the business owner may
require changes in associated projects.
5 FAM 675.3 IT Program/Project
Managers
(CT:IM-74; 05-03-2006)
Program and project managers are responsible for:
developing, modifying, or enhancing IT investments; understanding the users
mission and critical success factors (i.e., those activities and results that
must be realized if the users are to accomplish their mission); managing the
cost and schedule of the project; and measuring and reporting the quality of
the system as part of the CPIC process.
5 FAM 675.4 Enterprise Architects
(CT:IM-74; 05-03-2006)
Enterprise architects review the performance linkages and
measurement indicatorsdeveloped and categorized by the program sponsor and
program/project managersfor conformance to OMB guidance and directives and
alignment with the Joint Strategic Plan, Joint Performance Plan, or Bureau
Performance Plans. Any discrepancies or misalignments identified by the
enterprise architects are resolved via discussions with the program sponsor and
program/project managers. The enterprise architects are also responsible for
vetting the line of sight with the program sponsors and program/project
managers. Line of sight diagrams are created for each program (and in some
instances, specific projects and IT investments) by the enterprise architects
based on these discussions.
5 FAM 675.5 E-Government Program
Management Office (E-Gov PMO)
(CT:IM-74; 05-03-2006)
The E-Gov PMO was established by the Under Secretary for
Management to support the E-GovPB and to provide a single source to coordinate
and ensure completion of all program elements related to Department E-Gov/IT
projects. The E-Gov PMO manages the Departments CPIC process, including the
selection, control, and evaluation of IT investments to ensure compliance with
OMB guidelines for establishing, monitoring, and achieving performance measures
for all IT investments. (See 5 FAM 115.3-2).
5 FAM 675.6 E-Government Program
Board (E-GovPB)
(CT:IM-74; 05-03-2006)
The E-GovPB is the upper-level advisory entity to the
Under Secretary for Management that: addresses the full range of Department
E-Gov and IT investment portfolio and project management activities; ensures the
completion of all program elements related to the Departments IT investments
for meeting E-Government initiatives, and ensures that major milestones are met
throughout all stages of the CPIC process.
5 FAM 675.7 Deputy CIO for Business,
Management, and Planning (IRM/BMP)
(CT:IM-253; 11-26-2018)
IRM/BMP is responsible for the approval of all information
technology (IT) policy and procedures and their codification in the Foreign
Affairs Manual (FAM). Performance measures policy falls within this category.
5 FAM 675.8 Chief Financial Officer
(CFO)
(CT:IM-74; 05-03-2006)
The CFO ensures the development and maintenance of
integrated accounting and sound financial management systems that include
systematic measurement information in agency performance (see 5 FAM 115.5).
5 FAM 675.9 Chief Information
Officer (CIO)
(CT:IM-253; 11-26-2018)
The CIO is responsible for monitoring and evaluating the
performance of IT programs in accordance with applicable laws, regulations and
other directives, including the Presidents Management Agenda (PMA),
E-Government initiatives, and Office of Management and Budget (OMB) mandates
for accomplishment and accountability (see 40 U.S.C. 1425).
5 FAM 675.10 Chief Information
Security Officer (CISO)
(CT:IM-74; 05-03-2006)
The CISO carries out the CIOs security responsibilities
of developing and maintaining an agency-wide information security program (see 44
U.S.C. 3544(a)(3)(B)).
5 FAM 676 Why is security important in
measuring IT performance?
(CT:IM-258; 12-04-2018)
a. In addition to the Government Performance and
Results Modernization Act of 2010, the Federal Information Security
Modernization Act of 2014 directs agencies to perform annual independent
evaluations of information security programs (see 44 U.S.C. 3551). Bureau
executives, program managers, and system owners are required to identify,
track, and correct existing security deficiencies and implement processes that
guard against anticipated or identified threats and vulnerabilities.
b. OMB requires the Department to respond to OMBs
reporting requirements for information security performance measures, which
indicate how information and information systems are protected. OMB also tracks
budget submissions and links them with the Departments overall information
security performance. The measurement used to evaluate program performance and
legislative compliance is critical to the Departments ability to acquire and
maintain adequate funding for capital planning (see 5 FAM 677.2).
c. The Chief Information Security Officer (CISO) is
responsible for responding to OMB reporting requirements in the area of
information security and its overall performance.
5 FAM 677 what is the role of enterprise architecture (EA) in it performance measures?
(CT:IM-253; 11-26-2018)
a. The Office of the Chief Architect (IRM/BMP/OCA) has
developed a roadmap that identifies how State and USAID must transition from
how we operate today to how we want to operate in the future, to ensure that
the Departments business needs and mission are continuously met. Progress
toward these objectives must be tracked and reported, at the enterprise
architecture level, through a set of migration and transition plans that
specify the target performance metrics; and at the program level, through the
periodic one-on-one reviews of program performance metrics with the responsible
program sponsor, program manager, and project manager.
b. All IT investments must align with the strategic
objectives and goals stated in the Joint Strategic Plan, performance goals
cited in the Joint Performance Plan or Bureau Performance Plan, and measurement
categories and groupings in the performance reference model (PRM), and the
appropriate business areas, lines of business, and sub-functions in the Joint
Business Reference Model (BRM). (See E-Gov PMO Web site for links to PRM and
BRM.)
c. Managers must use the Federal Enterprise
Architecture Reference Models to view their approach to programs and projects
as follows:
(1) Establish the line of sight;
(2) Use the PRM to determine operationalized
measurement indicators; and
Identify reusable operationalized measurement
indicators that support the appropriate measurement areas. (See E-Gov PMO, PRM
model.)
5 FAM 677.1 What Is the Purpose of
the Federal Enterprise Architecture Performance Reference Model?
(CT:IM-74; 05-03-2006)
a. OMB created the Performance Reference Model (PRM) as
part of their Federal Enterprise Architecture (FEA) framework. The PRM
presents the links between the inputs (i.e., technology, human capital,
physical assets), and the internal business processes and activities necessary
to achieve desired business results and customer-centric outcomes. (See E-Gov
PMO.)
b. The PRM was developed to determine performance
measurement indicators for IT initiatives. The Department must use the agency
specific version of the PRM to assist in establishing performance measurement
indicators for IT projects. The Department must also use the PRM to:
(1) Categorize the measurement baselines and planned
improvements identified by the program sponsor and project manager;
(2) Use the Joint Business Reference Model (BRM) to
identify the lines of business and subfunctions that the program/initiative is
supporting; and,
(3) Identify which strategic objectives and goals from
the Joint Strategic Plan, Joint Performance Plan, or Bureau Performance Plan
must be accomplished.
c. Organizations must use the Federal Enterprise
Architecture Performance Reference Model (FEA PRM) as a guide to classify and
produce performance measures for all investments.
5 FAM 677.2 How Does Performance
Measurement Relate to the Capital Planning Investment and Control (CPIC)
Process and Enterprise Architecture (EA)?
(CT:IM-74; 05-03-2006)
a. The CPIC process is a systematic, standardized set
of procedures for selecting IT investments to fund, and subsequently manage,
their risks and returns. It is a continuous, integrated, management process
focused on achieving desired business outcomes and provides a mechanism for the
continuous selection, control, and evaluation of IT projects. The CPIC process
consists of three phases:
(1) The Select phase ensures that IT initiatives are
chosen that best support the Departments mission and strategic goals. The
Select phase consists of two components: the pre-select and the select
components. The pre-select component requires an initial business case be
made to justify the proposed investment before it is promoted to the select
component. The select phase requires establishing baselines and performance
measurement plans to properly align and conform with the joint strategic plan,
joint performance plan or bureau performance plan, and joint enterprise
architecture before being selected as a valuable initiative to fund;
(2) The Control phase ensures that IT initiatives
are conducted in a disciplined, well-managed, and consistent manner, which
promote the deliver of quality products, and result in initiatives that are
executed to the performance baseline (cost, schedule, scope) and anticipated
results (performance measures); and
(3) The Evaluate phase compares actual to expected
results once a project has been fully implemented (performance baseline and
performance measures).
NOTE: Both the control phase and the evaluate
phase collect information about a funded initiative that focuses on a
measurement of a programs performance, which is critical to identify any
necessary improvements or modifications, and the need for additional or
continued funding level.
b. All IT program and project managers must develop a
strong business case that:
(1) Provides supporting information to justify the
investment, and clearly defines the business requirements that are being
satisfied;
(2) Demonstrates all reasonable and practical
alternatives have been explored;
(3) Demonstrates no capabilities or services are
duplicated; and
(4) Ensures EA and IT policies, procedures, and
standards are adhered to and any necessary waivers have been approved.
c. A business case must also identify and specify the
relevant performance measures that the program sponsor, program manager, and
project manager understand and agree with. Such measures must justify the use
of the proposed IT resource, product, or service based on the required business
needs, strategic objectives and goals, performance goals, and mission.
d. Program and project managers must work with the
program sponsor of the IT initiative before, during, and after a sponsor
approves and funds the IT initiative. There must be a demonstration of how the
investment is in compliance with, and is working to satisfy, the target joint
enterprise architecture as part of the CPIC process.
5 FAM 678 What are the PRM performance
measurement areas?
5 FAM 678.1 Mission and Business
Results Measurement Area
(CT:IM-258; 12-04-2018)
a. The mission and business results measurement area
within the PRM is a major component of the line of sight that must indicate
what results, (i.e., outputs or outcomes) are expected to be achieved from the
IT investment.
b. The clearly defined outputs or outcomes should be
developed during the budget and strategic planning process prescribed under the
Government Performance and Results Modernization Act
of 2010. To identify the mission and
business results associated with an IT initiative, refer to the performance
goals identified in the Joint (DOS/USAID) Performance Plan.
c. To ensure the identified outputs are appropriately
aligned to what the programs actually do, the mission and business results measurement
area is driven by the BRM.
NOTE: The PRMs measurement categories are the same as
the business areas and lines of business identified in the Joint BRM. These
areas of the BRM identify the type and purpose of the activities conducted by
the Department and USAID.
5 FAM 678.2 Customer Results
Measurement Area
(CT:IM-74; 05-03-2006)
a. The customer results measurement area is a vital
part of the PRM that captures how well an enterprise-wide or specific process
within the Department is serving its customers and citizens. The purpose of
the customer results measurement area is to identify the customer relationship,
articulate how it can be measured, and identify subsequent improvements made
over time.
b. The customer results measurement indicator captured
in this measurement area will be associated with the most external customer of
the process or activity the IT initiative supports (e.g., citizens, businesses,
or other governments).
5 FAM 678.3 Processes and
Activities Measurement Area
(CT:IM-74; 05-03-2006)
The Process and Activities Measurement Area of the PRM
must identify and define the system, product, or service that will directly
result from the process an IT initiative supports. The processes and
activities measurement area is comprised of six categories: financial,
productivity and efficiency, cycle time and timeliness, quality, security and
privacy, and management and innovation.
5 FAM 678.4 Technology Measurement
Area
(CT:IM-74; 05-03-2006)
a. The Technology Measurement Area addresses key
elements of performance directly relating to the IT initiative. An IT initiative
can include applications, infrastructure, or services provided in support of a
process or program.
b. The technology measurement area is more relevant
when used with other measurement areas to see a full and accurate picture of
overall performance. As with all other measurement areas, the technology
measurement categories and groupings do not represent exhaustive lists.
Agencies may, and should, use additional technology measures as part of their
IT CPIC and systems development lifecycle processes.
c. While these IT-specific aspects of performance
(e.g., percent system availability) are important, they alone do not truly
assess the value of an IT initiative to overall performance.
d. The technology measurement area consists of seven
measurement categories. These categories are: financial, quality, efficiency,
information and data, reliability and availability, and effectiveness.
5 FAM 678.5 Human Capital
Measurement Area
(CT:IM-74; 05-03-2006)
The PRMs Human Capital Measurement Area is required by legislation
and best practices. Human capital measurement does not include specific
measurement categories at the present time. The Federal Enterprise
Architecture Program Management Office (FEA PMO) will begin to engage
organizations, such as the Office of Personnel Management and the Chief Human
Capital Officers Council, to work collaboratively to identify the key human
capital requirements and a set of practical and usable measurement indicators
in the Human Capital Measurement Area.
5 FAM 678.6 Other Fixed Assets
Measurement Area
(CT:IM-74; 05-03-2006)
The PRMs Fixed Assets Measurement Area continues to be
explored. A review of legislative requirements and best practices indicates
that capturing the performance of other fixed assets (e.g., vehicle fleets,
facilities, and other equipment) is critical. The PRM does not currently
include specific measurement categories for fixed assets. The FEA PMO will
engage officials knowledgeable about the management of other fixed assets as it
begins to improve the PRM.
5 FAM 679 unassigned