Back

Performance Auditing-The Three 'Es'

Muhammad Akram Khan, Director General,
Performance Audit Wing, Department
of the Auditor-General of Pakistan.

Introduction

Traditionally a corporate auditor has been concerned with the certification of accounts. He would apply necessary auditing procedures to express an opinion on the financial position of an organization. A Government auditor goes a step further. He also examines that the public expenditure has been incurred and revenue has been collected in accordance with rules, regulations, and under due authority. He sees that general canons of financial propriety have been observed by government departments and agencies.

Recently, the traditional scope of government auditing has undergone a change. The need for change has been felt for following reasons :-

It is being felt that an independent agency should review and report on the performance of government departments, agencies and their programmes. In response to the expectations of the public represen­tatives the Supreme Audit Institutions of a number of countries have expanded the scope of their examinations. The expansions in the scope has been differently termed in different countries. But the most common term is performance auditing.

2.    Definition

Performance auditing has been variously defined. A working definition is as follows:

"Performance auditing is an assessment of the activities of an organisation to see if the resources are being managed with due regard for economy, efficiency and effectiveness and accountability requirements are being met reasonably."

In this definition the key words are:

  1. assessment
  2. activities
  3. organisation
  4. resources
  5. management
  6. economy
  7. efficiency
  8. effectiveness
  9. accountability requirements
  10. reasonability

 

  1. Assessment: It means that the auditor formulates a judgement on the basis of relevant and reliable evidence. The judgement may contain a certain amount of subjectivity and hence may be contested by the auditee management. But to the extent it is based on agreed facts and is arrrived at after carrying out generally accepted auditing procedures it is treated as valid.
  2. Activities: Performance auditing extends to financial as well as non-financial activities of an organisation. It reviews the main operations of the organisation. Obviously, it implies that auditors would require some understanding of technical operations of the auditee. This may necessitate association of social or physical scientists or other specialists with the auditors.
  3. Organisation: Performance auditing takes an over­ view of the activities and functions of an organisation as a whole. In exceptional cases, it may review an isolated project or a programme as well. But for generalised conclusions it is necessary that the organisation is reviewed as a whole in its operational environment.
  4. Resources: The resources of an organisation consist of money, men, materials, and machines. Performance auditing reviews all these resources.
  5. Management: Management covers such functions as planning, organisation, resourcing, directing and controlling. Performance auditing reviews all these phases of management cycle. It is not concerned merely with the 'use' of resources.
  6. Economy: Review of economy is a primary element in performance auditing. Economy means acquiring resources at the lowest cost keeping in view the objectives of the organisation.
  7. Efficiency: Efficiency refers to the relationship of inputs and outputs. It relates to utilisation of resource.
  8. Effectiveness: It means the extent to which an organisation achieves its objectives.
  9. Accountability requirements: Public managers are responsible to carry out the policies of the legislature by translating them into appropriate programmes. Certain resources are provided to them to implement these programmes. They are expected to report back to the legislature about the results of their programmes. Traditionally, the report to the legislature is presented in the form of appropriation accounts, which is, as a matter of fact, a report of the financial results. With increased emphasis on receiving value for money and reducing waste in public spending, more and more legislatures are demanding reports on operations of government departments and agencies. The Supreme Audit Institutions have been authorised in a number of countries to report on the economy, efficiency and effectiveness of government operations. The report of the Supreme Audit Institutions comments on the adequacy of the executive department reports, so far as accountability requirements are concerned. It informs the legislature whether the reports of the executive operations provide a complete and satisfactory explanation of the results achieved, keeping in view the resources available to them. Where the reports of the executive are incomplete, ambiguous or unsatisfactory, the audit report makes a mention of it. Thus performance auditing acts as a catalyst to increase awareness of accountability among public managers.
  10. Reasonability: The performance auditing adopts the approach of a reasonable manager. It does not turn an oblique eye to the constraints of the management. Instead it takes into account the total environment in which executive departments operate. It does not base its judgement on hind-sight wisdom.

3.    The Three E's

Now we shall elaborate the concepts of economy, efficiency and effectiveness in relation to performance auditing.

(a)    Economy

While reviewing acquisition of resources for economy, the auditor tries to ascertain whether resources have been procured in the right amount, at right place, at right time and at right cost. Such a review presumes availability of acceptable standards. But the real life situation is that ready-made standards are not available. Therefore the auditor has to determine his criteria with reference to the primary objectives of the organisation. While determining the right amount of resources, he would see if the management laid down its needs clearly and, as far as possible, quantitatively. The assessment of needs should normally, lead to identification of requirements for which alternatives are analysed to determine minimum cost.

Right time to acquire a resource is also linked to the need to be fulfilled. The resource should be available to satisfy the need when it is required. It should neither make other resources 'wait' nor should wait for other resources. Therefore, the auditor reviews procedures to foresee demand, procurement and availability of resources.

Right place for a resource is the one where it is needed. The resources may be available where they are not needed. For example, there may be jobs which are unmanned and there may be men who do not have work. The auditor reviews the system for signaling the resource gaps.

Right Cost refers to lowest cost in the life-cycle of a resource. The lowest cost is determined in the following manner:

The above costs are estimated in the form of streams over the life-time of the asset. The cost streams are then discounted to the present using an appropriate discount rate.

It would be obvious that the above method is not a simple comparison of one-time capital cost, which is usually done in traditional auditing.

The calculation of minimum cost presumes that estimates of cost would be prepared for all available alternatives.

Economy should be perceived in relation to the achievement of other goals. Economy in an absolute sense may mean not to spend anything at all. But in a relative sense it has to be related to the achievement of goals of an organisation. Thus 'economy' means spending only that much which is barely essential to achieve the organizational goals. For example, it may be most economical to buy certain supplies from the lowest bidder but if it leads to interruption in the operations of the organization then it may not be the most economical manner of buying these supplies. Therefore, while reporting on economy the, auditor should keep in view effectiveness of the organisation in achieving its goals.

(b)    Efficiency: Efficiency is a difficult concept in government organisations. It presumes that standards of input and output are available. But in a large number of cases the standards are not available and the auditor has to work with the auditee management to determine agreed standards. The most commonly used standards, however, are planned outputs for given inputs laid down by the auditee itself. Where they are not available then other techniques are used to assess the level of efficiency. Some of the commonly used techniques are as follows:

(i)    Inter-authority Comparison: Comparison with similar authorities, delivering similar programmes and operating in comparable environments may be used to identify possible areas of savings. The areas for comparison could be the following:

(ii)    Internal Comparison: Comparison of costs between different services or facilities provided by the same authority at different locations or in different departments may lead to valuable insights. For example, repair costs by class of vehicle, or energy consumption by schools or residential units. A large range of performance indicators can be developed by the auditor from the internal data such as:

(iii)    Private Sector Comparison: In certain areas where private sector is also providing similar service, comparison with private sector could be useful. Examples are: architects, computer service, residential accommodation.

It is a sensitive area, however. Unions may be provoked. Moreover, the auditor may slip into policy area which is not his legitimate domain. Comparisons with private sector are useful only when the auditor distinguishes between the 'avoidable' and 'unavoidable' costs. A comparison with avoidable costs may lead to savings otherwise the exercise may be an academic one.

(iv)    Past Performance: Comparison with past years may also reveal potential areas for saving. The auditor should analyse the cost and output trends.

(v)    Targets: Targets set by the management for itself should be used as comparison measures.

Measurement of Efficiency

Measurement of efficiency involves an assessment of the following:

  1. Whether all levels of the organisation are aware of their goals?
  2. Whether there is a reliable system in place to monitor the output of each level?
  3. Whether the management receives regular reports on current levels of efficiency or services rendered?
  4. What are the measures taken by the management to improve its efficiency?
  5. What efficiency measures are feasible in the given situation?

Pre-requisites for the Measurement of Efficiency:

  1. The outputs should be uniform and repetitive. For example, filling in prescription by the pharmacist in a hospital. Non-uniform output like police programme, writing press briefs by Information Department etc., cannot be measured.
  2. There must be an agreed standard of performance against which actual output has to be measured. For example, in case of a school the number of class-contact hours for a teacher in a week should be laid down as a standard.
  3. If the standards do not exist then output may be measured with previous years. Sometimes measurements are compared between two geographical locations, or, comparison with private sector. It means data of past years should be available.

Advantage of efficiency measures:

  1. They help to control quality and quantity.
  2. They help to demonstrate achievements.
  3. They help to prepare budgets and plans.
  4. They provide a rational basis for pricing goods and services, v) They help in deciding' the level of service to be provided,
  5. They help in personnel performance appraisal.

Relationship with productivity:

For example, the time taken for producing 80 bags of cement is one machine-hour. This is productivity of the plant. This level of productivity may be 80% of the standard which is 100 bags per machine-hour. Efficiency is, therefore, measured in terms of rate of return of production, work done in a time span or unit cost.

(c)    Effectiveness: Review of effectiveness presumes existence of measurable objectives or outcomes of public programmes. These objectives or outcomes serve as criteria for the auditor. Expressing an opiniqn on the effectiveness is an important and vital breakthrough in the traditional scope of auditing. In certain countries performance auditng is known as effectiveness auditing, since issues of economy and effciency are adequately covered by traditional auditing.

Effectiveness auditing

Audit of effectiveness is perhaps the most important area for a performance auditor. Resources may have been obtained economically and efficiently but the auditee may not have achieved the objectives. The performance auditor shall see:

  1. whether the objectives have been defined by the auditee as precisely as practicable?
  2. how far the objectives have been based on facts and forecasts which appear reliable?
  3. to what extent the objectives have been achieved? iv) whether the policy instruments chosen to carry out objectives are based on detailed analysis of alternatives?
  4. whether the progress can be reasonably attributed to the activities and policies of t he auditee?
  5. whether the auditee has a satisfactory arrangement to monitor the effectiveness of a policy instrument?

In measuring effectiveness a distinction needs to be made between output and outcome. The output relates to the results of certain inputs within the organisation while outcome relates to the results external to the organisation. For example a certain project visualises to install a certain number of tube wells at a certain cost. The tube wells are the output of this project. But the tube wells are intended to lower the sub-soil water table by certain feet. The lowering of water-table is the outcome of the programme. Effectiveness auditing measures the outcomes of the public expenditure.

The question of identifying outcome of each project is rather ticklish one. It is not always obvious from the plan of the project in case the outcomes are identified in the plan it may not take into account all the outcomes (possibly negative) of a project. Therefore, the auditor is expected to exercise his ingenuity to identify all the outcomes of a project. One method of doing this is to imagine the impact the services of the project may have on the public.

Performance Measures for effectiveness

Appropriate performance measures to assess the effectiveness of projects are very difficult to devise. There are three main problems:

  1. Problem of jointness: Where a number of different policies may contribute to satisfying unmet needs. For example, educational standards may be affected by the size of classes, the quality of teachers and the. supply of equipment. It may be very difficult in practice to analyse the effect of individual policies.
  2. External factors: Sometimes factors outside the control of the management affect the outcome of a project or programme. For example, income and social status of the consumers.
  3. Cost: Sometimes programmes cannot be carried out in the most effective manner due to prohibitive cost. For example, it may be more effective to have more teachers than to provide more books. But the cost of the former may be prohibitively high.

Because of these difficulties in the measurement of effectiveness the auditor should move cautiously. The best method should be to devise performance indicators in consultation with the management. But the auditor should keep in view any national or international standards already available. Similarly, the objectives laid down in the plan may be taken as a bench mark for some of the outcomes. In no case should the auditor restrict his audit criteria to the original plan only.

Sources of Performance Measures: There are a number of sources of effectiveness measures, such as following:

  1. Citizen surveys,: especially in areas of social services, police, garbage collection etc. For reliable results, extreme care need be exercised in determining sample size and preparing questionnaires.
  2. Trained observer ratings: Trained observers are used to rate conditions in a particular area, using a pre-designated rating criteria such as street cleaning.
  3. Industry standards: Comparison of the results of a programme with prevalent standards in that area or with other similar programmes.
  4. Internal records: Review of internal records such as occupancy rate for residential homes, user complaints, etc. sometimes provides vital information for measuring' effectiveness.

Normally, the auditor is not expected to carry out the above exercise himself. His primary duty is to see if the auditee has a reliable system to evaluate its own effectiveness. In that case, he would focus his attention on the adequacy of these procedures. In case the auditee does not have a reliable mechanism to measure its own effectiveness the auditor may have to do it himself.

The main focus of effectiveness auditing is on the assessment of the extent to which an agency failed to attain the intended goals. But a simple explanation of possible reasons for shortfall in achievement of goals is not enough. For example, it is not enough to say that the agency could not achieve its goals because of cost over run or time over-run. Instead it is necessary that the auditor should analyse the reasons into following two categories.

  1. Factors that could have been influenced by the agency. They would emerge from an answer to the question: did the agency perform its task in the most appropriate way, seen in relation to the assignment that it intended to carry out?
  2. Factors that could not be influenced by the agency such as Government policy, applicable rules, socio­ political environment, appropriation of funds and availability of trained manpower, etc.

Effectiveness auditing, passess through the familiar planning, execution and reporting stages. At the planning stage the auditor decides about the audit approach, adequacy or inadequacy of the data, lays down the audit criteria and identifies issues of potential significance. Sometimes, disagreements develop with the management regarding the appropriateness of the audit criteria. In such a situation the matter is resolved either by referring to a higher authority or by allowing the auditor to go ahead independently or simply by abandoning the effectiveness audit restricting its scope to economy and efficiency aspects only At the execution stage the auditor reviews the adequacy of the management's system of effectiveness measurement. If no such system exists then the auditor develops a system of his own.

If the management has a system of effectiveness measurement then the auditor reviews its reports in the following manner:

  1. documents the management's system of effectiveness measurements;
  2. assesses the appropriateness of performance indicators;
  3. verifies the results and compares them with the appropriate standard;
  4. formulates an opinion on the effectiveness of management's system. (It is important to note that effectiveness measures differ from efficiency measures. For example, if a government department has the mandate to develop economically backward region, the efficiency measures would be the number of industries which moved to the backward region. But the effectiveness measures would relate to the figures of unemployment or income level).

If there does not exist (or exists but is weak) a system of effectiveness measurement then the auditor has to generate his or her own data. It may involve scrutiny of internal records or external surveys. Whatever method is chosen, the auditor should be careful about the following:

  1. externalities of the programme should be kept in view;
  2. cost of data collection should not make audit exercise uneconomical;
  3. the method should be agreeable to the management;
  4. preferably, management staff should be involved in the exercise.

4.    Relationship of Economy Efficiency and Effectiveness

The economy, efficiency and effectiveness aspects of an organization are closely interlinked. Sometimes, they trade-off with each other. For example, economy of post office department can be increased manifold by delivering mail once a week but it would cut down the effectiveness of the department to an unacceptable level. The auditor's role in arriving at the most desirable level of trade-off between three 'E's is rather difficult. A general approach is that the auditor should focus on the organisation's goals and try to see whether economy or efficiency measures are helpful in achieving those goals. In other words, the quality of performance should not be compromised in seeking greater economy or efficiency. On the other hand, if possible, effectiveness should increase by such measures.

5.    Concluding Remarks:

The above discussion highlights key concepts in performance auditing. The theory of performance auditng is fairly simple. But its practice presents difficult problems, some of which may bring serious difference of opinion. The auditors are expected to remain open to other point of view without sacrificing the objectivity, independence and rigour which their profession demands.