h2g2 Episode 10: It’s the log that counts

A Hitch-Hacker’s Guide to the Galaxy – Developing a Cyber Security Roadmap for Executive Leaders

In this blog series, I am looking at steps that your organisation can take to build a roadmap for navigating the complex world of cyber security and improving your cyber security posture.

There’s plenty of technical advice out there for helping security and IT teams who are responsible for delivering this for their organisations.  Where this advice is lacking is for executive leaders who may or may not have technical backgrounds but are responsible for managing the risk to their organisations and have to make key decisions to ensure they are protected.

This blog series aims to meet that need, and provide you with some tools to create a roadmap for your organisation to follow to deliver cyber security assurance.

Each post focuses on one aspect to consider in your planning, and each forms a part of the Cyber Security Assessment service which we offer to our member organisations in the UK Higher and Further Education sector, as well as customers within Local Government, Multi-Academy Trusts, Independent Schools and public and private Research and Innovation.  To find out more about this service, please contact your Relationship Manager, or contact us directly using the link above.

View all episodes.

Episode 10: It’s the log that counts

“Protect me from knowing what I don’t need to know. Protect me from even knowing that there are things to know that I don’t know. Protect me from knowing that I decided not to know about the things that I decided not to know about.”

Douglas Adams, A Hitchhiker’s Guide to the Galaxy

[ Reading time: 11 minutes ]

Digital footprints

Computers are good at remembering things.  I’m not talking about birthdays and anniversaries (although my phone always pings me useful reminders) or pub quiz answers (Googling strictly verboten).

No, the sort of routine events that make up everyday life.  When I bought a bus ticket.  The transaction for my morning coffee and pastry.  When I walked through the card-controlled entry system.  When I logged on to my laptop.  And on it goes.

Modern society has walked itself into surveillance by default in which enormous amounts of data about our everyday actions are routinely captured and processed.

These digital footprints are widely used for enforcement: evidence for parking fines, tracing missing persons, and tracking criminal activity.

And the IT systems running your organisation can do the same and more.

Every computer keeps track of what’s happening to it.  It’s called event logging.  If you’re interested, you can open the Event Viewer application in Windows and see for yourself.  Much of it won’t make much sense to the uninitiated.  By when it comes to cyber incidents, these event logs can be gold dust.

Log files are usually managed automatically by the computers themselves.  To keep them manageable, they are generally one of two types: when the log file gets to a certain size, the system starts a new one; or they are a fixed size and log entries are “rotated” by deleting the oldest ones to make space when necessary.  Windows computers default to using rotating logs although this can be changed.

Turning the clock back

A major question is how much log data do you need to retain?  There’s a bit of debate on this one.  The CIS Critical Security Controls specifies that you keep 90 days worth of logs.  If your organisation is subject to PCI DSS compliance for electronic cards payments, you’ll need a full 12 months of log data with 45 days worth immediately accessible.  Digital forensics specialists will tell you that an attacker can lie in waiting for many months before launching an attack, so logs going back beyond 90 days will be helpful in spotting the initial access method and what happened after that.

You need to make a risk-based judgment call on this.  Aim for 90 days as your minimum across the board (12 months if PCI DSS is an issue for you).  Store more if you can afford the storage space.

If at first you do succeed…

The next question is what do you record in the logs?  Many systems will decide automatically for you, but in some cases you can adjust the logging process in terms of what you record and in how much detail.

One of the fundamentals is security logins.  In Windows systems, you can choose to record successes or failure (or both).  Failures are useful because they will show up if an attacker is trying to crack a password.  Successes are just as important, though, because if an attacker has compromised an account and has the password, you want to be able to track where they’ve been.

Finding the needle in the haystack

Logging on individual systems is all well and good, but there’s a problem.  Cyber attackers use tools which scan networks and move laterally from one computer to another as they work.  So tracking the malicious activity becomes really hard, and time consuming.

The solution, of course, is to collate all the event logs in one place.  Having all the data together makes forensic analysis in the aftermath of an incident much easier.  Collating the data like this also allows it to act as an early warning system.

There are a number of ways to do this.  For Microsoft Windows systems, you can have a server running Windows Event Forwarding and configure all your other servers, desktops and laptops to send their log data to that server.  There are third-party tools (open source and commercial) which do the same thing, but can collate logs from other systems, like Apple Mac and Linux.

Either way, the problem now is that you have a huge collection of log data to search.  In reality, it’s like looking for a needle in a haystack.  Forensic analysis can take its time to work through the data, but to be effective as an early warning system, you need automated tools.

Siems a good idea

Security Incident and Event Management—SIEM, pronounced “seem”—tools are designed to solve this problem.  They can be installed on premises or in the cloud.  Common examples include Microsoft Sentinel, Splunk, LogRhythm, ManageEngine, FortiSIEM and GrayLog.

Not all SIEMs have the same capabilities, but they all do the basics of ingesting log data from various sources to provide a single viewpoint.

The more sophisticated of them apply algorithms to sift through the events looking for abnormal or anomalous patterns of user or computer behaviour.  This is important, because it can stop an attack in its tracks.  Especially if the SIEM tool also has the capability to intervene when it detects malicious activity.

What’s not to like?  Well, it’ll come as no surprise that there’s a price tag for this capability.

It comes in 2 flavours.  Firstly, in licensing costs.  On top of the cost of the SIEM solution itself, you’ll pay more to unlock advanced features.  Secondly, for cloud-based services, there are typically charges for the volume of logs ingested, and may be additional charges for non-standard log types or automated investigation and response.

There is an increasing trend towards a cloud-based subscription or “pay as you go” model for IT service procurement, which on the balance sheet features as OpEx (or revenue) rather than CapEx.  This can be challenging for budgeting and create headaches for financial control, so needs careful management.  We recommend trialling before committing.

Keeping time

Have you ever wondered how it is that your computer or phone always shows the right time?  Well, it’s not down to a finely tuned quartz mechanism.  Instead, they check in regularly with a network time server to synchronise their clocks.  For mobile devices, that time server is usually across the internet somewhere.  For the Windows servers and workstations in your organisation, its usually your network “domain controller”.  The domain controller itself should be configured to synchronise with an internet time server.

Why do we care?  Well, when it comes to event logs, getting the time right on different device matters.  Because cyber incidents occur at CPU speeds, any difference in log times might change our understanding of what happened when.

So, it’s important that the domain controller is synchronised to an internet time source, so that events from mobile devices correspond correctly with those from servers and workstations.  And while Windows devices are taken care of, other systems—like CCTV and door access control systems, alongside Apple and Linux based systems—will need to be manually configured to use your domain controller as an authoritative time source.

Keeping tracks

Cyber attackers will always look to cover their tracks by deleting logs once they’re done.  A centralised log server or SIEM makes that harder, but will itself be a major target for attackers, especially if you are running your own server on your own network.  You need to put tight controls on your centralised log servers (whether these are on-premises or in the cloud) to minimise access to only authorised personnel.

A Final [Deep] Thought

In the next episode of A Hitch-Hacker’s Guide to the Galaxy, we’ll be looking at Disaster Recovery planning.  “Be the master of disaster.”

For now, you can take useful steps forward by reviewing your organisation’s log management processes.  Do you have logging enabled on all your key line of business systems?  Are they synchronised to an accurate local time server?  Are you logging login successes as well as failures?  Do you have a centralised logging server or cloud-hosted SIEM?  How well protected is it?  Do you have an early warning capability?


James Bisset is a Cyber Security Specialist at Jisc.  He has over 25 years experience working in IT leadership and management in the UK education sector. He is a Certified Information Systems Security Professional, Certified Cloud Security Professional, and is a member of the GIAC Advisory Board.

Leave a Reply

Your email address will not be published. Required fields are marked *