In order to get a better perspective,
we'll go through a demo where we'll parse SMB logs using regular expressions.
To make the process of crafting the pattern easy,
I'll be teaching you how to use CyberChef to craft patterns
which are highlighted on the log samples,
taking the pain out of troubleshooting them later.
I'll start off this demo in the Terminal section and launching Python,
and I will import the log_samples module where I have
a log sample from an SMB log file.
What we notice is that this does not have a standardized structure,
both colon and pipe are used as separators.
This is where we need to use regular expressions to carve
out all of the fields individually.
While I can attempt to do this in the terminal, I want to show you a better way.
So I'll copy this log line and I will launch CyberChef and paste the log entry.
Using the search option,
I will look for a regular expression and then drop it in the Recipe section.
CyberChef has a couple of built‑in regular
expressions that we can use straightaway.
For example, let's try the one with IPv4.
Nice.
We can see right away the IP address in the log entry being highlighted.
But for our purposes,
we have to define a custom regular expression to properly identify all fields.
So, I will switch to the user‑defined instead.
The first element of the regular expression is the caret sign,
which identifies the start of the string.
I will make use of name groups as this will help us
to retrieve the fields directly.
The first one being the timestamp, noted as ts.
This can be represented as three groups of two digits separated by colons,
and this is the first named group.
Then I will match the space with a \s, followed by a colon,
and followed by another whitespace.
The next group is the client hostname.
This can contain lower and uppercase letters,
as well as numbers and the dash or minus character,
which needs to be escaped with the backslash.
In Parallel, we can see these fields getting highlighted.
After matching the pipe character,
the named group corresponding to the client IP can be identified by
four groups of three digits separated by a dot.
To match the name of the share, I will use the same pattern as for the hostname.
The plus sign is to match one or more characters defining the set.
The operation can be identified just with lowercase and uppercase characters.
The ok can be matched,
but it doesn't require to be part of a named group since
only successful actions are logged.
The last named group is the path.
Here, the path can contain any characters because in some instances,
the folder is also part of the path and the rename log entry contains two paths.
And to match the end of the log entry, the dollar sign is used.
And now that the pattern is done, I will copy this and return to VS Code.
Similar to the previous parsers,
I will define a new parser for the SMB log entry and
paste in the pattern into a variable.
The r in front of the string is used to treat this as a raw string.
I will then use the re.search function and pass in the
pattern and the log entry as parameters.
This will be assigned to the log_data variable.
To get the dictionary composed of all the named groups,
I will use the groupdict function and return r.
To test this, I will jump into the terminal and import this new parser.
I will use the previous log sample to test this.
The timestamp field is again marked as a string,
which needs to be converted to a datetime object.
To experiment, I will assign this to a variable called ts.
I will now import the date and time module,
and I will use the strptime function where I will pass in the timestamp,
as well as the format string composed of %H for hours,
colon, %M for minutes, colon, %S for seconds.
This is known as a format string,
and it tells the function how to extract the information from the timestamp.
And we can see it returns a datetime object,
with the date of the 1st of January, 1900,
as this was not part of the original timestamp,
but it can be modified later.
So let's implement this in the parser as well.
There is one more element to addressing the parser.
I have a second sample of the log file,
which logs the rename operation where this actually logs when someone
renames a file or it moves it to a different folder.
To tackle this, I will check if the operation is a rename,
and if so,
split the path based on the pipe character and only
use the second or last value,
which is the name the file was renamed to or the path
where the file has been moved to.
Now let's test these modifications in the terminal.
I will reimport the samples and the parseSmb function.
I will run the function using the first log entry example
to check the date is parsed correctly, and it is for the specific time.
And running the function again for the rename log entry type,
we notice that the second part is logged.
One more important aspect is the documentation on the
format codes for date and times.
This will be really useful if you want to parse various date and time formats.