survey-results

Audit Metrics and Performance Measures—Survey Results
Audit Policies andPractices Committee
Objective
Identifyand compile internal and external metrics and performance measures used in the Federal audit community for evaluating audit quality and timeliness.Compilefactors and best practices used in annual audit planning.
2
Approach
Collaboratively developed survey questions onInternal Reporting – 6 questionsExternal Reporting – 2 questionsAnnual Planning – 10 questionsSolicited input from FAEC community in May 2013Received responses from 34 Audit organizations.
3
Defining a Quality Audit
Answers objectivesProvides timelyfindings andactionable recommendationsAddresses critical and high-riskareasAdheres toqualityassuranceguidance (e.g.,Yellow Book, internal andpeerreviews)Involvesclient satisfaction surveys to obtainfeedback.
4
Audit Quality and Timeliness
5
8 metrics and performance measures surveyedConsiderable feedback provided
Audit Cycle Time
Tracked and monitoredIncluded inemployee performance standards.
6
Audit Process Efficiency
Internalguide to evaluateauditsin processPerformance metric and tracking ofTimeliness of specific phasesStaff workdaysDirect time chargesReturnoninvestment
7
High Risk Activities/Programs
Agencystrategicgoals/prioritiesGAOhigh-riskareasIGmanagementchallengesEnterprise-levelriskassessmentInputfrom senior organizationalleadersAnnualrisk-based program assessmentsPercentageof planned performance audits addresshighriskareas
8
Congressional and OMB Mandates
9
Establish metrics to ensure audits meetstatutorymandateddeadlinesEstablish individual performance standards
Financial Benefits
Tracks monetary benefits with specific goalsReport at least $30M in potential monetary benefits each fiscal yearMeeta return on investment goal of at least $10: $1, $8: $1Ensure 10% of published reports identify potential monetarybenefitsEnsure 60% of performance audits include recommendations leading to demonstrable cost savings and/or funds put to betteruseMonitorsbenefitsviamanagementinformationsystemPerformance metricbased on the dollar savings identified during the year.Reviews actual realized benefits during follow-upengagementsReports financial benefits only as a workload measure;no goal
10
Significant Identifiable Benefits
Monitorsrecommendation implementation and follows upwhenimplementationhas notoccurred by target date ofcompletionDiscusses audit work in context of organization’s qualitativegoalsIncludes corporate goals such as:70% of past recommendations implemented within two years of the fiscal year in which the report wasissued60% of performance audits should include recommendations leading to demonstrable cost savings, program efficiencies, and/or funds put to betteruse
11
Management Decisionsand Implementation
Information obtained beforethe report isissuedInformation tracked viainternal management system.Allopen audit recommendationsare discussed monthly withaudit resolutionofficialsMetrics established forreceipt of comments and actionstakenRequest comments and recommendation responses within 30 days of the draft report issuance.Achieve a 5-year average implementation rate of 85% for accepted recommendations.Obtain final management decisions and/or target implementation dates within 90 days of report issuance.Write recommendations so organization can complete them within 6-9 months.Receive action plans within 60 day of final report issuance.Reporttwice a year on all unimplemented recommendations past 6 months of original target date toagency senior leadership.
12
Established Milestones
Weeklyexception reportfor projectsthathave notmetmilestonesCommunicatestatus of high impact audits during monthly senior managementmeetingEstablishes an estimated amount of audits to complete in the fiscalyearIncorporates an organization milestone goal fordeliverablesDeliver 60% of audits with draft reports within 300 elapseddaysIssue reports within 270 days of auditstartComplete 80% of projects with a year.Finalize 90% of assignments within 30 days of established final report milestonesFinish audits within one year from entranceconferenceConclude audits based on audit scope: Standard audit is 11 months from entrance conference to final report; limited scope is 2.5 to 10 months from entrance to finalreport
13
Measure Office Performance Balanced Scorecard Approach
Try to do a balanced approach for financial, process efficiency, and program impact, but no formal scorecard
14
Setting Targets for Qualityand Timeliness
Totalcycle timeFrom initiation to report issuance -- 8 months for non complex audits to 13 months for complex audits.Total elapsed days for each projectContract/Grant audits:5 days on-site & 5 days for survey and report completion.Recommendations concurred with byAgency80% areacceptedPercent of open Recommendationsimplemented95% of past recommendations are implemented.Number of Audit reports produced; percentage of reports containing one or more recommendationsFinancial Benefits & Non financialbenefitsReturn on InvestmentPercentage of audit completed with established timeframes (90%, 75%)Complete 90% of audits within 15% of established budget and within 30 days of report milestones;75% of audits address high risk activities
15
Ability to Sustain Funding
NoFurloughs this FiscalyearMoneyis available for travel and training at a reduced rate.Workplan focused on audits requiring less travel; on track to meet CPE requirements.Additional funding from ARRA and Hurricane Sandy relief.
16
Measuring Client Satisfaction
17
External Reporting
18
External ReportingDollar Amount Reported
Report Issuance 24Amounts Agreed to 13Amounts Agency Agreed to Collect 19 report at issuance and agreed to
19
Annual Planning
One associated this with assessing the risk environment to develop risk categories
20
Annual Planning
Number of intended reviewsTimelinessRatio of requested to internally-generated auditsCoverage of Agency priorities/missionCoverage of Management ChallengesCoverage of GAO High Risk AreasAmount of Direct Staff time applied to critical and high impact projectsNumber of reports issued aligned by Management Challenges
21
Annual Planning
22
Annual Planning
Other Factors ConsideredCongressional or Agency leadership requests,New programs,Audit history and risk with existing programsFollow up on prior recommendationsStakeholder inputRisk AssessmentResource AvailabilityBalance of coverage across Agency programsTiming to ensure most value to client
23
Annual Planning
Other Factors (cont.)Hotline complaintsLarge grant recipients/dollar programsOMB A-123 and FMFIA assessmentsGovernment-wide initiativesStatutory requirementsMedia interests/emphasisSpecial funding allocations (e.g., Recovery Act, Disaster Oversight, etc.)Investigative referralsImpact on the delivery of services and benefits to recipientsPublic interest
24
Annual Plan Drivers
25
Plan Modification
Ad Hoc (as needed) 22Scheduled update 6Not Typically Modified 6
26
Thanks to the Team
John Needham, SBAStephen Dingbaum, NRCTerri Alvarado, Army AuditAnn Eilers, CommerceJane Mintz, CommerceJulie Marlowe, SECGil Harden, USDA
27
Final Thoughts
QuestionsThanks forproviding information
28