FISMA Compliance: Metrics or Management?

The Federal government consistently gets poor marks in annual IT risk management reviews required by the Federal Information Security Management Act of 2002. A Government Computer News editorial by Wyatt Kash discusses the ramifications of the latest findings, and suggests that the source of the Government’s persistent and pervasive information security troubles is this: Information security management practices are not measured.That’s a compelling argument, given that government isn’t generally held to rigorous management performance criteria. And when agencies do have criteria in place, the measures are often distorted to present a picture of accomplishment where there is none. One agency, for example, measures the number of help desk calls as a success criterion; more help desk calls equals better service. Clearly, the focus should be on reducing the number service calls. But government managers unknowledgeable about IT management principles simply look at the numbers and assume that more is better.

The GCN article suggests that the audit guide used by the payment card industry (PCI) offers the measures of information security that NIST guidance, and the FISMA law itself, lacks. PCI audit guidelines have fairly specific success criteria, including:

• Maintain a firewall configuration to protect data;
• Don’t use vendor-supplied defaults for system passwords and other security parameters;
• Protect stored data;
• Encrypt transmission of sensitive information across public networks;
• Use and regularly update antivirus software;
• Restrict physical and logical access to cardholder data;
• Assign a unique identifier to each person with computer access;
• Track and monitor all access to network resources and cardholder data;
• Regularly test security systems and processes.

Jorgen T. Lazo, an IT Analyst with the Federal Reserve Board, rejects the GCN editorial’s suggestion that PCI metrics will solve security problems. “While security is the issue, the burden of managing our information systems is the real root of the problem.” Lazo goes on to say that IT systems passing security checklists are not necessarily secure-we have plenty of evidence of that in the number of reported breaches of systems that were supposedly secure. The problem lies in the way we design, build, and operate IT systems. Information security practices and techniques should be “baked into” IT delivery.

Both points of view are credible. IT management does need to measure security processes (Kash). Information security is not just operational; it’s the whole package—a strategic, tactical and operational concern. (Lazo)

Control Objectives for Information and Related Technology (COBIT) meets both goals. COBIT provides a strategic IT governance framework that defines management objectives for IT processes, and it defines performance measures informing management whether those objectives are being met. In COBIT, security objectives and measures are pervasive throughout every IT management activity: planning and organizing, acquisition, implementation, delivery and support, monitoring, and evaluating performance.

Let’s take an example. COBIT specifies that risk management is a significant component of planning and organizing the IT activity. Risk management requires processes that:

• Establish an IT risk management framework that is aligned to the enterprise’s risk management framework;
• Establish the context in which the risk assessment framework is applied to ensure appropriate outcomes;
• Identify events (important and realistic threats that exploit a significant vulnerability) with a potential negative impact on the goals or operations of the enterprise, including business, regulatory, legal, technology, trading partner, human resources and operational aspects;
• Determine the nature of risk event impacts, and record and maintain relevant risks in a risk registry;
• Assess the likelihood and impact of all identified risks on a recurrent basis, using qualitative and quantitative methods;
• Determine the likelihood and impact associated with inherent and residual risk individually, by category and on a portfolio basis;
• Develop and maintain a risk response process designed to ensure that cost-effective controls mitigate exposure to risks on a continuing basis; the risk response process should identify risk strategies, determine associated responsibilities, and consider risk tolerance levels;
• Prioritize and plan the control activities at all levels to implement the risk responses identified as necessary, including identification of costs, benefits and responsibility for execution;
• Ensure that committed actions are owned by the affected process owner(s);
• Monitor execution of the plans, and report on any deviations to senior management.

The measures of success toward meeting the goals of the above processes include:

• Percent of critical IT objectives covered by risk assessment;
• Percent of IT risk assessments integrated in the IT risk assessment approach;
• Percent of identified critical IT events that have been assessed;
• Number of newly identified IT risks (compared to previous exercise);
• Number of significant incidents caused by risks that were not identified by the risk assessment process;
• Percent of identified critical IT risks with an action plan developed;
• Percent of IT budget spent on risk management (assessment and mitigation) activities;
• Frequency of review of the IT risk management process;
• Percent of approved risk assessments;
• Number of actioned risk monitoring reports within the agreed-upon frequency;
• Percent of identified IT events used in risk assessments;
• Percent of risk management action plans approved for implementation.

Note that, in contrast to PCI, none of the success criteria mentions any specific security activity like maintaining firewalls or assigning a unique identifier to each person with computer access. Instead, COBIT supplies IT management with measures to evaluate how well risks have been identified, assessed, and managed. If maintaining a firewall won’t address a risk to IT assets, then it shouldn’t be a priority. But, for example, if there is a significant risk to information in transit, then encrypting the transmission of sensitive information across public networks is a good idea.

Mr. Kash accurately points to the need to measure how well you’re addressing security activities. But, as Mr. Lazo points out, addressing security from the bottom up won’t provide an adequate payoff. Yes, PCI provides a framework for good security practice; but it’s still just a checklist. Managers need to think strategically, not just operationally, about the real risks to IT assets. In short, management needs to apply governance. COBIT gives them the tools to do just that.

►