>
Advanced
Training Module
Cost Estimation
Status
10
September 2003
2
Advanced Training
Module
Background
Task
assigned to SAI USA at 11th meeting of the INTOSAI Standing
Committee on IT Audit in New Delhi, India.
SAI USA
had previously submitted a paper for the INTO IT Journal on cost estimating
SAI USA
has been using cost estimating methodologies as a part of value for
money and technical audits
3
Advanced Training
Module
Status
SAI USA
has completed development of the Cost Estimation and Analysis training
module
Will provide
a brief overview at this time
Will be
using it internally at the GAO to train personnel
Will also
be developing a cost estimation and analysis guide in the next year
Would
like to thank the committee for giving us the opportunity to work on
this effort
Advanced
Training Module
Overview
Karen
Richey
Senior
Cost Analyst
5
Advanced Training
Module
Cost Estimation & Analysis
Outline
Introduction
to Cost Estimating
Life Cycle
Costs
Data Collection
Data Analysis
Cost Estimating
Methodologies
Expert
Opinion, Analogy, Parametric, Engineering, and Actual
Software
Cost Modeling
Cross-checks
and Validation
Risk and
Sensitivity Analysis
Documentation
Requirements
Cost Estimating
Challenges
Cost Estimating
Auditor Checklists
6
Advanced Training
Module
Introduction
to Cost Estimating
Basic
introduction to cost estimating
Links
to places where additional material can be found
Limitations
on what cost estimating can and cannot do in an audit
Types
of estimation methodologies
7
Advanced Training
Module
Life Cycle
Costs
Definition
and discussion of the 00cope00of an estimate
Discussion
on how a life cycle cost can be used
Introduction
of a Cost Element Structure (details later) used to collect various
costs for a project
Example
CES
8
Advanced Training
Module
Example
CES
1.0 Investment
Phase 2.0 System Operations & Support Phase
1.1
Program Management 2.1 System Management
1.2
Concept Exploration 2.2 Annual Operations (supplies/spares)
1.3
System Development 2.3 Hardware Maintenance
1.3.1 System Design &
Specification 2.4
Software Maintenance
1.3.2 Development Prototype
and Test Site Investment 2.5
Outsource Provider Support
1.3.3 Software Development 2.6 Data Maintenance
1.3.4Training 2.7 Site Operations (personnel,
training,etc.)
1.3.6 Facilities
1.4
System Procurement
1.4.1 Deployment Hardware
1.4.2 Deployment Software
1.4.3 Initial Documentation
1.4.4Logistics Support Equipment
1.4.5 Initial Spares
1.4.6 Warranties
1.5
Outsource Provider Investment
1.6
System Initiation, Implementation & Fielding
1.7
Upgrade (Pre-Planned Product Improvement (P3I))
1.8
Disposal Costs
9
Advanced Training
Module
Data Required
Discussion
of data required to generate a cost estimates
Direct
(engineering, materials) vs. indirect (support, rent, insurance) costs
Data sources
00primary and secondary (reports, contractor site visits)
Data validity
and integrity
Normalizing
data for inflation
10
Advanced Training
Module
Cost Estimating
Methodologies
Expert
Opinion,
Analogy,
Parametric,
Engineering,
and
Actual
11
Advanced Training
Module
Considerations
Choice
of methodology is dependent upon
Type of
system
Software,
hardware, etc
Phase
of program
Development,
Production, Support
Available
data
Historical
data points from earlier system versions or similar system
Technical
parameters of system
12
Advanced Training
Module
Detailed
Description for Each Methodology
Overview
Strengths
and limitations (Pros & Cons)
Steps
required for each methodology
Simple
case study
13
Advanced Training
Module
Method
Example - Parametric
Utilizes
statistical techniques called Cost Estimating Relationships (CER).
Relates
a dependent variable (cost) to one or more independent variables
Based
on specific factors that have a high correlation to total cost
Number
of software lines of code (SLOC) or function points,
Square
feet for office floor space,
Number
of floors in a high rise building for cabling estimates,
Database
size, etc.
Can be
used prior to development.
14
Advanced Training
Module
Method
Example - Parametric
Pros:
Can be excellent predictors when implemented correctly
Once created, CERs are fast and simple to use
Easily changed
Useful early on in a program
Objective
Cons:
Often lack of data on software intensive systems for statistically
significant CER
Does not provide access to subtle changes
Top level; lower level may be not visible
Need to be properly validated and relevant to system
15
Advanced Training
Module
Method
Example - Parametric
Historical
data has enabled the organization to have a previously developed CER
to estimate a new IT system based on SLOC.
Cost =
SLOC * 25 $/SLOC
The CER
is based on systems ranging from 1,000,000 to 3,000,000 SLOC.
You have
estimated 2,600,000 SLOC for new system
Cost =
2,600,000 * $25 = $65M
16
Advanced Training
Module
Method
Example - Parametric
Cost estimators
can develop their own CERs or they can use existing commercial cost
models.
Various
Software cost estimating models will be discussed next
Learning
curves 00specialized type of CER
CERs can
be cost to cost or cost to non-cost
Cost
to Cost: e.g., Manufacturing costs are 1.5 times Quality Assurance
costs
Cost to
Non-Cost: $/pound, or engineering hours/# of engineering drawings
yields hours/drawing metric that can be applied to new program
Factors
and ratios are also examples of parametric estimating.
17
Advanced Training
Module
Method
Example - Parametric
Reliable,
normalized data is most important for CER development.
Must determine
range of data for which the CER is valid.
Useful
at any stage in a program.
Typically
CERs are the main cost estimating methodology in early stages of a program.
In
later stages of a program, CERs serve as a cross check to other methods
Must be
logically sound as well as statistically sound.
High
correlation (r2 = 0.75 or higher) for goodness of fit test
18
Advanced Training
Module
Method
Example - Parametric
Different
statistical techniques may be used to judge the quality of the CER.
Least
squares best fit (regression analysis, or the ability to predict one
variable on the basis of the knowledge of another variable)
Multiple
regression (a change in the dependent variable can be explained by more
than one independent variable)
Curvilinear
regression (relationship between dependent and independent variable
is not liner, but based on a curve)
Learning
curve (describe how costs decrease as the quantity of an item increases)
19
Advanced Training
Module
Method
Example 00Parametric (Learning Curve)
The first
100 tasks of an installation took 10 hours per task and the next 100
averaged 8 hours per task. Thus, the learning curve would be calculated
as follows:
Learning
curve = 8 hours per task/10 hours per task = 0.8
Implies
an 80% learning curve, meaning an improvement of 20% occurred between
the first 100 tasks and next 200 tasks
20
Advanced Training
Module
Method
Example 00Parametric (Software Cost Models)
Software
costs as a percentage of total system costs continues to increase while
associated hardware costs decrease.
Accurately
capturing software costs can be difficult and cost overruns often occur
as a result of software requirements being difficult to estimate and
track.
21
Advanced Training
Module
Method
Example 00Parametric (Software Cost Models)
Software
estimating problems occur most often because of the:
Inability
to accurately size a software project,
Inability
to accurately specify a software development and support environment,
Improper
assessment of staffing levels and skills, and
Lack of
well-defined requirements for the specific software activity being estimated
22
Advanced Training
Module
Method
Example - Parametric (Software Cost Models)
There
are numerous software cost models
COCOMO
COSTXPERT
SLIM
SEER
Costar,
REVIC, etc.
23
Advanced Training
Module
Method
Example - Parametric (Software Cost Models)
Software
cost models typically:
Require
software lines of code (SLOC) to estimate cost
Can be
used to estimate size given typical applications (business, aerospace,
real-time control, etc)
Have various
parameters which modify cost,
For
example, skill, tool set, programming language, operating system, processes
used for development, etc.
Approximate
the real world through a combination of
Statistical
analysis of historical data,
Informal/Intuitive
analysis of rules of thumb based on experience
Contain
the following features
Development
and Support cost calculations, time-phasing to spread estimates by year,
and inflation to convert to budget year dollars
24
Advanced Training
Module
Advanced Training
concludes with brief discussion on:
Risk and
Sensitivity Analysis
Documentation
Requirements
Cost Estimating
Challenges
Cost Estimating
Auditor Checklists
25
Advanced Training
Module
Thank
You
download Advanced Training Module Cost Estimation
