Home /

Detecting a Security Breach

Q. What is my role as a systems administrator?

A. The role of a systems administrator is very important in the prevention, and detection, of security breaches. As well, users/customers complement the systems administrator in helping to detect breaches (for example, unavailable services or a defaced website).

Q. What can I do as a systems administrator right now?

A. The first thing you can do is check your systems. It is important to not assume that, even though a system currently appears to be running correctly, it is safe or uncompromised. It is entirely possible that the system was compromised before you took charge of it, or that it was compromised in such a way as to not alert you.

Tools such as chckrootkit can test various aspects of the system for compromise. Windows users can try the Microsoft Baseline Security Analyzer which will scan for insecure configurations among other things. Another Microsoft tool is windows update. While this won't test for vulnerabilities already in the system, it will at least be able to provide one with the most up to date operating system patches.

  • Chkrootkit: locally checks for signs of rootkit
  • MBSA: Microsoft Baseline Security Analyzer
  • Windows update: online extension of Windows that helps you to keep your computer up-to-date

Q. What is the most important thing a systems administrator can do?

A. By far, the most important thing a sysadmin can do is patch, Patch, PATCH. This goes back to being proactive. When one has machines under his or her charge, they have to make sure that they stay on top of vendor notices of security problems. Security Focus's Threat Management System website reports an average latency of 99 days between notification of a vulnerability and the release of exploit code. This gives you some time, but not much as many of the high profile exploits of the last year had exploit code released the same day.

Another very important thing for sysadmins to do is to turn of un-needed services at the time of machine install. There's simply no reason for a desktop system to be running a service such as IIS or, with the prevalence of secure protocols such as ssh, a print server to be running an ftp server.

  • Information about ITS-supported operating systems is available on ITS' Desktop Standards web page

Q. How should I respond to, and report, a breach?

A. If you suspect that a machine in your charge has been breached, the first thing you should do is report the incident to UCSC IT Security.

STOP - DO NOT INVESTIGATE FURTHER ON YOUR OWN
UCSC IT Security will take the responsibility of determining if there really was a breach (note: it is possible that in your hunt for evidence of a breach you may destroy evidence which could be needed later in the event of a formal investigation or possible legal action. It is even possible that by hunting for evidence, yourself, you may be out of compliance with UC policy.)

We need to stress the importance of timely notification. We're sure everyone is aware of the passage of California Senate Bill 1386. As a recap, this bill states that in the event of a compromise or suspected compromise of a machine which contains unencrypted personally identifiable information (PII), the persons whose information resides on that system must be notified immediately. Immediately isn't defined, but in the case which brought about this bill, six weeks passed between discovery and notification. There are some exceptions to the notification, but whether or not your particular situation falls under one of them is for law enforcement to decide, not you. Failure to notify the security team of a breach could potentially leave the university in legal trouble. So let us know.

Q. What other steps can I take to make this easier?

A. If you've done nothing from the start to help yourself detect and prevent breaches, there are few things that you can do to determine whether or not your systems have been compromised. These can be quite time consuming - but there are things you can do that will make a system breach much more noticeable and hence, discovering and reporting them easier.

  • Logging: For Unix systems, enabling appropriate logging (for example, sending all critical messages to one log file, all warning messages to another, and so on) and then monitoring those log files is a good start at reducing the signal to noise ratio that all sysadmins must deal with. From there, a log monitoring tool such as Logcheck can be used to parse system log files and send email alerts, further reducing the amount of time you must spend in order glean important information from your busy logs.
  • Checksums: Using tools such as Tripwire, which take cryptographic checksums of system binaries and configuration files and notify you of differences on subsequent runs can greatly minimize the amount of time required to detect intrusions.
  • Consider registering with the Center for Internet Security to download and read a security benchmark guide on your platform. These documents have detailed guides and instructions on how to secure your platforms and operating systems. They also include files and locations that should be monitored to detect breaches. Center for Internet Security Benchmark Download

Q. What is UCSC IT Security doing to help recognize breaches?

A. Currently, UCSC IT Security has several measures in place to help prevent and inform us (and you) about enterprise level intrusions and intrusion attempts. For example:

  • N-IDS: We have network intrusion detection systems (N-IDS) that look at network traffic for malicious traffic and send notifications if they encounter such.
  • H-IDS: We are running host based intrusion detection systems on various systems.
  • Routers: We enforce various access controls and filtering on our routers.
  • Netflows: We analyze network flows for both detection and investigation.
  • Firewalls: We provide firewalling services for systems co-located within the Data Center (from architecture and design to implementation, monitoring and management).

In the event that we become aware of an incident, either external or internal, we coordinate with the sysadmins in charge of the campus machines in question in order to determine the extent of the breach if any as well as come up with a plan to mitigate the damage and fix the hole.