Back

A way to Measure Performance in Information Technology:

....N. Nagarajan I.A.&A.S.

Introduction:

Information Technology (IT) is undoubtedly the fastest growing sector in the world. The fast evolving sphere of IT has created tremendous opportunities for all the sectors of the country. This has helped in the increased speed of processing, increased storage capacity of the system, and an increased availability of new software to perform wide variety of tasks and operations. Thus fueling the growth in the software sector too.

IT provides tremendous scope for improving operations, increasing or adding value to services and performance of new services.

Significant trends are:

1) Development of IT based networks largely powered by the development of internet, intranet, telecom and network computers. This development has created new avenues for several information services involving storage, searching, collecting and supplying information. Growth of network has helped in spread of Electronic Data Interchange (EDI), e-mail, e-commerce and the latest to follow e-governance.
2) New Technologies have emerged that allows more efficient information storage. Date mining and warehousing are becoming increasingly important. IT allows departments to co-operate by opening up the possibility of data integration and matching.
3) Emergence of object oriented tools for system development has made development of complex IT based systems quicker and simpler.
4) Transactions by the firms are increasingly being processed electronically and transactions are being received electronically. Now this has caught up with the Government also and the latest trend e-governance is associated with it.
5) Expert and automated management systems have emerged and is playing an important role in helping/guiding the managers. This equips the managers with better system for managing, supervising and controlling operations. 
6) Growth of mobile PCs and access to networks allow remote access to central computer and data resources altering the divisions between home and work place.
7) With the growth of IT considerable effort is being directed towards development of new and improved security measures allowing use of smart cards for obtaining different services.

IT Investment can be broadly classified as 

a) General investment for IT
b) Investment in the development and implementation of specific IT projects in different business areas to handle specific organizational tasks and activities such as financial accounting, management information processing, document processing, inventory management, project management, provisions of services etc.
c) Investment in IT related infrastructure such as communication networks, data warehousing, training and up gradation of skills, hardware and software support system and IT services etc. This investment spawn across organizations and with in organization across units.

Normally any investment made needs to be evaluated in order to assess whether the investment has achieved its desired results. The use of IT in an operation finds manifestation in considerable investment in IT by the organization. This poses novel challenges for the auditor as often the use of IT has considerable impact not only on the organizations operations and expenditure but also on the delivery of business benefits and achievements of business goals and objectives.

It would be necessary to measure the performance of IT investment from the standpoint of whether the resultant business benefits are delivered on time, at reasonable cost and were of an appropriate quality. Since the use of IT is widely believed to have the capacity to add value to business and the organization and make the organization work efficiently, effectively, openly and methodically, auditors shall have the additional task of determining whether the additional attributes are achieved.

Case study:

In order to justify the investment in IT and to analyse whether the investment in IT has achieved its desired results any organization that has invested substantially in IT must analyse the benefits derived due to implementation of the IT system.

For the purpose of this case study an organization that has adopted EDI system of processing customs declarations has been selected and performance measures are analysed.

Brief about the system:

Customs department of a country in African region has introduced the computerized customs procedures for processing of declarations and manifests with the implementation of Customs Management System (CMS) in an Electronic Data Interchange (EDI) environment.

CMS procedure in brief:

1) Declarant prepares Customs Declaration (CUSDECS) in their front end system.
2) Front end system of the trader calculates tax etc and sends CUSDECS to customs department mailbox at Network Service (NS) their third party service providers Value Added Network (VAN).
3) CUSDECS retrieved by CMS and processed by the CMS server in customs department.
4) After successful processing it is validated by CMS and sent for verification by verification department or for delivery.
5) Criteria for verification are set by a team and they are regularly reviewed.

For conducting the performance audit of IT projects the most important step is identification of evidence of performance.

Benefits due to implementation of IT System:

Potential benefits can be classified under three different categories like

1) Direct financial benefits,
2) Direct non financial benefits and 
3) Indirect benefits.

More work done, Greater range of information, fewer mistakes, easy, cheap and fast communication and improved management control may be considered as Direct financial benefits. Fewer mistakes, more reliable information, improved turnaround time, quick storage retrieval of information and better quality of output can be considered as direct non financial benefits. Improved image can be considered as an indirect benefit when benefits can not be measured in cash terms, it is important that objectives are set and their achievements measured. Such output and performance measures should be clearly set up to evaluate the performance of IT systems and its impact on the department.

Observation:

It was observed during preliminary survey that the department has not conducted any performance analysis even after ten years of its implementation of CMS system. So, audit has prepared a statement of performance measures like;

The department has set targets for only 13 of the 22 required performance parameters and the department does not even have a Quality service committee. Though the targets set are more relaxed than the comparable systems the achievement was far below the targets set.

Risk:

If the performance is not improved the Department runs the risk of disruption in operation and the whole objective of computerization (I,e Customer-oriented with an aim for excellence in quality of service, result oriented, with a keenness to aim at higher productivity) may remain unfulfilled. This may result in poor service and also loss of customer satisfaction and confidence.

Recommendation:

It was recommended that performance indicators should be identified for non financial benefits. Benefits should be assessed to include strategic importance. Risks should be assessed and the following targets can be set to improve its performance.

THE STANDARDS (Ref-1)

The department has to set up a Quality Service Committee to constantly monitor the service standards. The officer-in-charge of each section is also responsible to ensure that targets are met and improvements are made.

The Department has stated that the various line managers are being briefed to establish standards proposed and noted the point of audit (i,e) need for a quality service committee favorably.

Summary:

Introduction of new technology introduces new problems. This is true of any technology more so of the emerging and powerful technologies. One of the peculiar problem that arise is the absence of standards or benchmarks. The use of new technology becomes opinionated and constructive directions are missing.

Several audit and academic research have revealed that IT projects often experience time and cost over runs and do not perform in the way envisaged. In order to analyse the efficiency and effectiveness of an IT investment parameters must be set for measurement of performance.

Key Performance Indicators:

• Should have measures to analyse how well the process is performing

• Should be able to Predict the probability of success or failure in the future,

• Must be process oriented, but IT driven

• Should Focus on the process and learning dimensions of the Balanced Business Scorecard

• Should be expressed in precisely measurable terms 

• Should help in improving the IT process when measured and acted upon

• Should focus on those resources identified as the most important for this process.

Department should identify performance indicators and efforts must be made to set performance indicators for the standards stated above. Organisation with strong performance indicators would do an outstanding job at the vision, achieving targets, providing clear performance feedbacks and establishing a clear consequent management mechanism

Reference: 1. Singapore Customs & Excise department performance targets on Tradenet, its EDI System of processing Customs Declarations (source its website) due permission has been obtained to use this portion in my article.