DNS traffic analysis is a complex subject and best
explained through the power of examples,
so let's focus on this in this next demo.
We will count occurrences of unique domains based on the DNS traffic logs.
We will extract the root domain to reduce false positives.
After which, we will perform similarity analysis on the least visited domains.
To perform occurrence analysis on the DNS logs a similar
technique is used to the one used to calculate the number of
HTTP requests for unique UIDs.
I'll name this function getDnsAnomalies and have one
parameter to specify the path to the log file.
I will get the generator for the log file and save it to the log_file variable.
I'll create a new Counter dictionary.
And next,
I will iterate through all of the log entries in the file
and also make use of a try/except block.
For each of the log entries,
I will parse them and save the output in the log_data variable.
I will extract the dns_query from the query field and update
the Counter with the value of the dns_query.
At the end of the function I will add a return statement with the Counter value.
So let's see how this works.
I am opening the terminal using Ctrl+J, or command+J on a Mac,
and start up the Python repl.
I will import the newly created function.
Next,
I will use a variable called domains to store the Counter value
returned by the function for the Zeek DNS log.
Using array slicing I will select the first 5 elements.
And notice, there is a lot of CDN‑related requests.
In contrast, I will also display the 5 least common domains.
The problem that we start to notice right away is that some
very popular domains have a lot of subdomains,
which make our analysis a bit harder.
One solution is to focus only on the root domain,
which is composed of the domain name and the top‑level domain.
To do this, let's consider the following example.
We have a domain called app.pluralsight.com.
In order to extract the root domain,
first we need to split this by the dot character,
and we get an array with three elements.
Next, using array slicing, we'll select the last two elements.
And we only get pluralsight and com.
And finally, we need to join them together.
Perfect, we got the desired result.
To test this on an edge case,
let's change the domain name to something more longer,
like something.really.important.app.pluralsight.com,
and re‑execute the code.
We notice we still get pluralsight.com.
Now let's implement this in our function.
Since this function returns a dictionary,
let's implement a display and the reporting function separately.
And we'll call this printDnsAnomalies, and we'll have the same path parameter.
For now, we'll keep it simple,
we'll execute the getDnsAnomalies function and save
the results in the domains variable.
And next, we'll iterate through each one of the entries and print it.
Next,
we can focus on conducting the long‑tail analysis and focus on
the domains which have a very low occurrence.
This can be something like phishing pages.
So rather than returning the dictionary, we can return an array.
This will be called least_common,
and it will be assigned the value of the Counter dictionary for
which the most_common function is called,
and it will only get the least 10 elements,
which'll be the least common.
Also,
I need to change the return value to the least_common
rather than the Counter itself.
So let's test this as well.
I will only import the printDnsAnomalies function and
execute it using the path to the DNS log.
Now, if we pay close attention, we notice the globonamtics.com domain.
This is a very similar domain to globomantics.com,
so it could have been registered and used for malicious purposes.
So the next question comes as,
how can we codify similarity analysis into our function?
Well, there is a class called SequenceMatcher from the difflib module.
I will import this as sm.
The first parameter is set to None because we want to run a
comparison between two strings and we don't want to highlight
any characters that should be ignored.
Then I want to compare two strings.
And for this I'll call the ratio function,
which returns the value of 1, meaning they are identical.
Now, let me compare two different strings,
yellow and blue, for which the similarity is only .2.
But what if I want to compare two similar strings,
like yellow and yell0w, but with a 0 instead of the o.
This is interesting, the value is .83.
To get a percentage,
I can multiply this number by 100 and use the round
function to get 83% similarity.
Now let's implement this into our main function.
First, we'll import SequenceMatcher from the difflib module.
Because the least_common is an array of multiple arrays of two elements,
the domain and the occurrence,
I'll create a new array of dictionaries called domain_anomalies.
I will then iterate through the least_common domains.
And for each one, I will create a new dictionary with a domain name,
the occurrence, and the similarity score.
To compare the least visited domains,
I will add in a new argument to the function called similar_domain,
and we'll set the default value to globomantics.com.
So,
similarity is calculated for each of the least
visited domains and globomantics.com.
But this can be changed to anything else like the domain of your company
or a very popular service like Microsoft or Google.
Once this dictionary's created I will append this to the domain_anomalies array.
Now,
using format strings I will improve how the results are displayed and
ensure that the important fields are displayed,
like the domain, like the domain name, occurrence,
and similarity.
One important aspect, which I overlooked,
is to ensure that they getDnsAnomalies should return the new array called
domain_anomalies instead of the least_common domains.
Testing this in the terminal reveals the desired results.
We can further improve the quality of the results by sorting the
list descending based on the similarity score.
And we can do this with the array.sort function and
the itemgetter to specify the key.
Testing this again provides a better view.
For analysis purposes, once we identify a suspicious domain name,
we might still want to get the full query,
including any relevant subdomains.
To do this we can build another function called getDnsQueries.
This uses as arguments the path and the domain name.
Similar to the other functions, we iterate through all of the log_file entries,
parse them,
and then check if the domain argument is a substring of the query value,
and if so, we'll print the full domain name,
as well as the corresponding IP address based on the answers field.
Let's test this out for the globonamtics.com domain.
And this is interesting,
c2.globonamtics.com points to an IP address which we have
identified previously as the C2 server.