Audit: DHS cyber center lacks metrics

The 24-hour cybersecurity watch center at Department of Homeland Security is performing all the functions Congress wanted, but DHS has no way of measuring how well it is aligning with the guiding principles lawmakers set down, a new audit says.

Congress’s investigative arm, the Government Accountability Office, published the audit of the National Cybersecurity and Communications Integration Center, or NCCIC, on Wednesday. At the NCCIC, DHS officials sit side-by-side with representatives of state and local governments and the private sector owners of vital national industries to monitor the security of their computer networks.

The report includes the results of a survey randomly administered to the NCCIC’s “customers” — the governmental and private sector organizations that receive its bulletins or use one of the other 42 kinds of products and services the center offers.  Because of a low response rate, auditors say the results are not generalizable to the center’s customers as a whole. But they note that respondents “expressed generally favorable views of [NCCIC’s] activities.”

For instance, between 35 percent  and 45 percent found a cross-section of NCCIC’s products and services “extremely” or “very” effective, timely, relevant, or actionable; 25-30 percent found them “moderately” effective, timely, relevant, or actionable; and only 12-18 percent “slightly” or “not at all” effective, timely, relevant, or actionable.

The center reported that it spent about $480 million on cybersecurity-related activities during the three fiscal years 2014-16 — roughly $160 million a year. According to auditors, this included about 260 personnel in 2014-15 and just over 300 in 2016.

The NCCIC was set up in 2009 but not authorized by Congress until 2014. The National Cybersecurity Protection Act of that year, and the 2015 Cybersecurity Act the following year, established it, according to GAO, “as a federal civilian interface for sharing information related to cybersecurity risks, incidents, analysis, and warnings with federal and nonfederal entities.” The other 10 functions it was assigned by the two acts included:

• Providing “shared situational awareness to enable real-time actions to address cybersecurity risks and incidents.”

• Coordinating information-sharing on “cyber threat indicators, defensive measures, cybersecurity risks and incidents across the federal government.”

• Integrating, analyzing and distributing that information to federal and nonfederal partners.

• Facilitating coordination and cooperation between industrial sectors to address cybersecurity risks and incidents “that may be related or could have consequential impacts across multiple sectors.”

•  Providing “timely technical assistance, risk management support, and incident response capabilities … which may include attribution, mitigation, and remediation.”

• Providing “recommendations on security and resilience measures.”

• Engaging with international partners.

• Participating in national exercises run by DHS.

• Coordinating with DHS’ Office of Emergency Communications “regarding cyber incidents to public safety communications” like 911 call centers.

“NCCIC has taken steps to perform each of its 11 statutorily required cybersecurity functions,” write the auditors, saying the center has “developed a variety of [43] products and services” to fulfill these functions “including those related to analyzing and sharing cyber information, facilitating coordination among federal and nonfederal partners, and conducting technical assistance and exercises.”

But the National Cybersecurity Protection Act of 2014 also mandated the center to carry out those 11 functions “in alignment with” nine principles, and GAO says the department has no way of measuring the center’s adherence to those. The nine principles are:

• “Ensure that timely, actionable, and relevant information related to risks, incidents, and analysis is shared.”

• Ensure that when appropriate, that information “is integrated with other information and tailored to a sector.”

• Ensure that its activities are prioritized based on the level of risk.

• “Ensure that industry sector-specific, academic, and national laboratory expertise is sought and receives appropriate consideration.”

• “Ensure that continuous, collaborative, and inclusive coordination occurs across sectors.”

• Ensure that the center works “to develop and use mechanisms for sharing information … that are technology-neutral, interoperable, real-time, cost effective, and resilient.”

• “Ensure that the center works with other agencies to reduce unnecessarily duplicative sharing of information related to cybersecurity …”

• Ensure that cybersecurity information is “appropriately safeguarded against unauthorized access.”

• “Ensure that activities conducted comply with all policies, regulations, and laws that protect the privacy and civil liberties of United States persons.”

“NCCIC has limited assurance that it is fully meeting statutory requirements and efficiently performing its cybersecurity functions because it has not completely evaluated its performance against the principles or addressed the impediments to performing its cybersecurity functions,” the auditors conclude.

They said they found some evidence that the NCCIC was adhering to the nine principles — for example using the so-called Traffic Light Protocol to classify its informational bulletins depending on the distribution they were allowed, in line with principle eight: safeguarding cybersecurity information against unauthorized access.

But there were also principles were it was impossible to say if NCCIC was in alignment — for example it had no metric to assess the timeliness or effectiveness of the vulnerability assessments it conducted.

“Until the center determines the applicability of the [nine] implementing principles for all of its [11] functions and develops the metrics and methods necessary to ensure that the principles are met, it will not be positioned to ensure that NCCIC is effectively meeting its statutory requirements,” the auditors conclude.