Back

PROPOSAL:

Response to the Performance Audit of the AQMD for the FY Ending June 1997

SYNOPSIS:

Thompson, Cobb, Bazilio & Associates have completed their audit of the AQMD’s performance and issued their report. The auditors presented their findings to the Board at its June 12, 1998 meeting. Staff has prepared the attached responses to the findings and recommendations.

RECOMMENDED ACTION:

Approve this response to the AQMD’s performance audit for the Fiscal Years 1994-95, 1995-96, 1996-97.

Barry R. Wallerstein, D.Env.
Acting Executive Officer

Background

Health & Safety Code Section 40453 requires the AQMD to contract with a qualified firm once every three years to conduct a performance audit of the AQMD’s activities. The first audit was awarded in July 1991; the second audit was awarded in July 1994. At the June 1997 meeting, the Board approved a contract with Thompson, Cobb, Bazilio & Associates to conduct the performance audit for FY 1994-95, 1995-96, 1996-97.

The objective of the performance audit was to evaluate AQMD policies, procedures, and productivity. The evaluation took into consideration the findings of the performance audit done in 1994 and recommendations adopted thereafter. The activities within the AQMD Work Program for FY ending June 30, 1997 were the foundation for the performance audit.

The performance audit was conducted in accordance with Generally Accepted Governmental Auditing Standards (GAGAS), the guidelines set forth in a statement issued by the Comptroller General of the United States entitled, Government Auditing Standards, Standards for Audit of Governmental Organizations, Programs, Activities and Functions. In the context of GAGAS, the performance audit focussed on reviewing compliance with laws and regulations, economy and efficiency of operations, and effectiveness in achieving program results.

Summary

Thompson, Cobb, Bazilio & Associates has completed their audit of the AQMD’s performance and issued their report. The auditors presented their findings to the Governing Board at its June 12, 1998 meeting. A total of thirteen findings were noted in the areas of key processes, management issues, compliance with laws and regulations, and achievement.

Staff has prepared the attached responses to the findings and recommendations. In many instances, staff has begun to implement the recommendations. As required by law the report, along with AQMD’s action plan to implement the recommendations, will be submitted to CARB and the state legislature.

Attachment

Response to Findings and Recommendations from AQMD performance audit.

/ / /

AQMD Actions to Implement Prior Audit Recommendations

1. Finding: Of the 11 prior audit recommendations – including those to expand small business assistance, continue economic development/business retention program, and ensure CEQA requirements are met in a timely manner – the auditors find that the AQMD has implemented all, with some additional action needed for three items. Recommendation • AQMD has implemented the recommendation to develop a program oriented budget summary and more meaningful performance measures; however, further actions to refine the number and nature of performance measures would increase their usefulness as a management tool. • AQMD has taken specific actions to address its diminishing revenues and continued mandates. Opportunities for increased efficiencies identified in all sections of this report are pertinent to this issue. • The Strategic Systems Plan for Information Management has been completed but has not been formally approved. However, major portions of the plan have been substantially implemented and implementation continues.

Response Staff will incorporate this recommendation into the development of the FY 1999-2000 Budget. A tracking system to capture and report performance measures will be implemented. Discussions will be initiated with the AQMD Budget Advisory Committee in August/September regarding this matter. At its June 12, 1998 meeting, the Governing Board awarded a contract to evaluate the AQMD’s fee structure and approved the release of an RFP to evaluate streamlining the AQMD’s permit system. The Strategic Systems Plan for Information Management is being updated and will be presented to the Board for approval at its August 1998 meeting.

Key Processes

2. Finding: Rulemaking The rulemaking process is designed to develop fair requirements and that the process includes significant public notice, meetings, workshops and hearings. Process could be improved by involving more representatives of affected small businesses. Recommendation AQMD should make a concerted effort to inform small businesses or people unfamiliar with the AQMD of changes to rules. Provide businesses with descriptive information on the rules and contact points for answering questions. Distributing information on rule requirements through material and equipment suppliers would appear to be an effective means of communication.

Response Staff will implement this recommendation through developing an enhanced outreach plan for each rulemaking. This plan will include measures to identify all affected parties by the rulemaking including, but not limited to, meeting with trade organizations, equipment suppliers, and small business groups to identify affected companies. For the information to be distributed to small businesses, staff will revise its public notices to provide information that is more descriptive and key contact points. A revised public notice format will be developed and discussed with the Small Business & Local Government and Stationary Source Committees at their August meetings.

3. Finding: Permitting The permit application and renewal process generally works satisfactorily. The overall system could be improved by adopting some additional management controls and implementing computer system enhancements. Changes in the Certified Permit Processor program are needed to reduce problem applications and increase permit processing efficiencies. Recommendation • Establish procedures to log all checks received by the AQMD for daily deposit. Ensure adequate separation of duties between access to customer accounts and access to checks. • AQMD should fully integrate its accounts receivable and CLASS system, which will enable the automation of refund check processing. • AQMD management should identify and evaluate what information engineers should provide to the CPP Coordinator on all permits processed by CPPs to determine whether the program is effective and should be continued. (continued next page) • AQMD should make changes to the permit Pre-Screening and CLASS computer systems to provide: (1) keyword search capability; (2) computer program edit functions; and (3) ability to enter application and permit numbers as part of each transaction. Also recommend AQMD establish a tracking system in pre-screening to monitor status of permit applications.

Response Check handling procedures were amended and reviewed by the auditors, incorporating their recommendations. The programming necessary to automate the refund process was initiated in May 1998, with implementation scheduled for October 1998. AQMD has formed a new Permit Streamlining Task Force consisting of Board members and industry and environmental representatives. The task force is evaluating potential changes to enhance the CPP program and a draft work plan with regard to CPP program enhancements, which includes, among others, improved tracking, and consistency and quality control for CPP applications. (continued next page) Currently the AQMD has several contracts in progress that will result in significant enhancements to the permitting system. Software programs under development track applications for facility and non-facility permits and Title V permits as they pass through the permitting process. Permit applications that are on hold awaiting submittal of additional information or correct fees will be monitored and tracked prior to acceptance or rejection. This capability at the prescreening level is included in the Permits Enhancements task order scheduled for completion in the 3rd Quarter 1998. Additional proposed permitting system enhancements include a keyword search feature that provides the capability to search equipment for determining fees based on various parameters. The modified permit system will compute fees and incorporate the pre-screening program edit function recommended by the auditor. Contract status is either awaiting system activation, in the user acceptance-testing phase or in development with completion by the end of this calendar year.

4. Finding: Inspection A substantial amount of the total inspection resources are deployed based on call-in complaints or in response to required annual inspections for certain types of facilities. A more strategic approach based on targeting facilities can produce a huge benefit. Redundant information systems have been established to provide information that is not available in the CLASS system. Recommendation AQMD should redirect its inspections based on an annual strategic plan that targets areas that are expected to produce the greatest air quality benefits.

Response Staff concurs with this recommendation and will provide such a plan to the Governing Board by September 1998 and on an annual basis thereafter.

5. Finding: Customer Satisfaction Overall AQMD customers are generally satisfied, 73%, with permit processing, permit renewals and annual emission reporting, with 11% dissatisfied. Some dissatisfaction was expressed regarding emission reporting forms. Recommendation AQMD should have an ongoing program for evaluating and improving customer service. Specific levels of customer satisfaction should be targeted, measured, and reported.

Response AQMD staff is re-establishing its customer evaluation survey program. All results will be measured and reported. In addition, customer service training for AQMD staff has been reactivated.

Other Management Issues

6. Finding: Organizational Structure AQMD has undergone significant down-sizing, going from 1,172 employees in 1992 to 764 in 1997 which has lead to imbalances in the mix of management, supervisor, secretarial/support vs. line staff. Recommendation AQMD should move toward a "flatter" organization with less levels of supervision and hierarchy. Organizational span of control imbalances and workload should be examined and adjusted.	Response The AQMD is in the process of reorganizing its permitting, inspection and rulemaking functions which will result in a flatter organization with a better mix of supervisor to line staff. This reorganization will be implemented during the new fiscal year.
7. Finding: Use of Facilities AQMD has significant amount of office space that is vacant. The AQMD has not aggressively marketed the space due to restrictions imposed by federal tax laws that limit private use. Revenue generated from sale of El Monte site could reduce existing bond debt resulting in interest savings of $4.2 million and add greater flexibility in leasing vacant space to private sector. Recommendation Reevaluate office space with use of a space planner to gain additional efficiencies and free more space for leasing. Retain professional broker to market available space. Use proceeds from sale of El Monte to retire portion of Diamond Bar debt thereby increasing amount of vacant space that could be leased to private sector.	Response At its June 12, 1998 meeting the Governing Board authorized the redemption of Diamond Bar Installment Sale Bonds with the proceeds from the sale of its El Monte property. In addition, staff is exploring various options available to the Board regarding the sale or lease of a portion of its Diamond Bar property. Currently, 5,500 sq. ft. of space is leased to the federal government’s Treasury Department generating approximately $103,000 in annual lease revenue. An RFP for marketing services relating to the lease of additional vacant office space is scheduled for release in July 1998.
8. Finding: Use of AB2766 Motor Vehicle Registration Funds Cost effectiveness criteria and methodology are needed to ensure the most effective use of AB2766 funds by local government. Identification of the most cost effective projects/methodologies could increase program effectiveness by assisting cities in selecting projects. Current technical assistance resources may not be adequate to increase program effectiveness. Recommendation AQMD should coordinate with state ARB to identify most cost effective projects for local government; provide program guidance to local governments on cost effective projects; encourage the development of enforceable cost effectiveness criteria; coordinate regional projects among multiple jurisdictions; and provide the technical air quality expertise that cities do not have.	Response As California Air Resources Board has finalized cost-effectiveness criteria and companion methodology, AQMD staff will be incorporating ARB guidance in a newly created handbook for local governments to be distributed in FY 98/99. Expanded technical services are included in a recently released RFP to provide help to Local Jurisdictions utilizing AB 2766 Subvention Funds. In addition, AQMD is already working with ARB staff to: • Revise the Criteria and Guidelines section of the AB 2766 Subvention Handbook to identify cost-effective projects and to refine the current analytical tools needed to select cost-effective projects; • Coordinate a series of forums for local governments within the South Coast Basin; • Convene working groups consisting of technical staff from AQMD and cities and counties within the District; • Create a central depository to showcase successful projects that local governments have initiated and established.

9. Finding: Contracting Procedures AQMD does not have procedures to ensure that a price or cost analysis is performed when only one qualified bidder is obtained through competitive bidding. When a contractor selection process is used that differs from the normal procedure, a formal justification or assessment is not provided. Personnel performing the technical assessments of contractor proposals should not have access to information on proposed cost to ensure balance between technical and cost considerations. Task orders need to be carried out in the manner they were approved. The AQMD has expanded its sources for employment law and is performing more work in-house. Recommendation AQMD should perform price or cost analysis for open-bid solicitations of $50,000 or more when there is only one qualified bidder. The use of contractor selection formulas other than the normal procedure should be justified. During the evaluation of the technical portion of a contractor proposal, panel should not have access to information on proposed costs. Ensure that the specified guidelines for evaluating and awarding task orders are followed. AQMD should continue its use of expanded sources for employment law and use additional outside counsel where cost effective.

Response The auditor’s recommendations concerning price/cost analysis of single bids resulting from competitive solicitations, justification for use of non-standard contractor selection formulas, and evaluation and award procedures will be incorporated into the next revision of the AQMD’s Procurement Policy scheduled for October 1998. These recommendations are also being implemented at this time.

10. Finding: Information Technology • Need to improve communications between IM and users. • CLASS software could be improved by adding needed features; improve software documentation and training. • Need to address user concerns, institute regular means of communication. • AQMD Intranet has not been effectively exploited or developed. Infrastructure and expertise are already in place. • The Intranet has tremendous potential to improve workflow at the AQMD. Recommendation • AQMD should ensure that systems development needs, status, and progress are routinely identified, publicized, and reviewed by all parties. • AQMD should exploit its Intranet resources to aid in overcoming communications problems, establishing and documenting appropriate operating procedures and performance measurement systems. • AQMD Intranet development activities should be formally budgeted and managed. (continued next page) • All systems development should take into account the central role to be played by the Intranet. • AQMD management should adopt the Strategic Plan.

Response Information Management (IM) submits a Major Projects Board letter monthly listing major projects, cost and status information for review by all parties. Also, IM has standardized on Microsoft Project to provide task level management and status of all development projects. Staff agrees that greater use of the Intranet would improve internal communications. Therefore, FY 98-99 funds have been budgeted for Intranet development projects that will be issued as task orders during the first six months of FY 98-99. IM has established a formal line item for Internet/Intranet development in its FY 98-99 budget. These projects will be managed via the outsource task order process. (continued next page) AQMD moved the first Intranet workflow application (Board Agenda Tracking System) into production in May 1998. AQMD has targeted Intranet workflow application development as a major goal in its Strategic Plan and continues to evaluate Web front-end applicability and compatibility for AQMD systems. The Strategic Plan for Information Management is being updated and will be presented to the Board for approval at its August 1998 meeting.

11. Finding: Performance Budgeting and Measurements The AQMD has far too many performance measures which focus on activity rather than key results that link to AQMD’s overall mission. Recommendation AQMD should overhaul the existing system, ensuring that the new measures: • Flow from key organizational goals • Are limited in number • Developed with active participation by those responsible • Are results oriented.

Response As mentioned under finding number one, staff will incorporate this recommendation into the development of the FY 1999-2000 Budget. A tracking system to capture and report performance measures will be implemented. Work on this item will begin in the August/September time frame.

Compliance with Laws and Regulations

12. Finding The AQMD was in compliance with the requirements of all but one of the state and federal laws reviewed. For the Air Toxic "Hot Spots" program: • The AQMD has made steady progress in implementing the required elements of the program • Some interim deadlines were not met • The program appears to have reduced public exposure to toxics as some facilities voluntarily reduced or eliminated toxic emissions to avoid exceeding program thresholds. Recommendation The AQMD should continue its efforts to streamline the process for the review and approval of the remaining HRAs.

Response At its May 8, 1998 meeting, the Governing Board approved the release of an RFP to contract with an outside firm to assist in finalizing the remaining HRAs. The selected firm will assist in approving the remaining HRAs within the next 18 months. Existing staff would require an additional three years, given current resource levels.

Achievement

13. Finding: The AQMD’s performance, with respect to the attainment of air quality objectives directly under its control, has been outstanding and the magnitude of its accomplishments is large. AQMD still has room for improvement in its performance and efficiency. Future improvements in air quality will depend on an accurate quantification of mobile emissions and on the implementation of stringent methods for their control. Recommendation • Some responsibility for mobile source control within the SCAB should be shifted from the ARB to the SCAQMD or a strong working partnership between the two agencies with respect to mobile source control strategies and enforcement should be developed. • Super mobile-source emitters should become the major focus of future enforcement efforts. • Stationary source enforcement and compliance programs need to be maintained to ensure continued success. • AQMD computer models of the SCAB should be upgraded to reflect the most current understanding of the chemistry and physics of air pollution. • A more responsive and interactive scientific advisory program with a strong peer review process needs to be established. • Members of the engineering, science, and modeling groups at the AQMD should be encouraged to publish more of their work in the open scientific literature.

Response Staff concurs that added emphasis on mobile source controls is needed for clean air attainment. Staff will continue to work with CARB and U. S. EPA to ensure that appropriate efforts are taken to further control mobile source emissions. Staff will discuss this item with CARB and U. S. EPA who have principal authority in this area. Staff concurs and has reflected this priority in the FY 1998-99 budget. Staff concurs and has included this item in the Year 2000 AQMP workplan. Staff concurs. The workplan approach for the year 2000 AQMP includes greater outside peer review. Staff concurs.

/ / /