This is major one, lesson six,
hedging your biases.
This lesson, we have two main
objectives.
The first is to go over the
importance of collaboratively
assessing attack mappings
is going to cover a bit of why some
of that step five is so important
from the attack mapping process.
Then I'm going to get into some
about analyst and source biases
and some ways to hedge against them.
Step five of the process we gave,
which is to compare with other
analysts, can be really important in
doing attack mapping or frankly,
any other intelligence analysis.
This can help hedge against
analyst biases.
So I'll be getting into a number of
different reasons why
different analysts might come up
with different answers.
But everyone has a different set of
experiences and background that
they're drawing upon as they do
intelligence analysis.
So where one analyst may
see not application layer
protocol and another may see
custom command and control protocol,
it's important to figure out why
these differences exist,
be consistent and how you map and
apply techniques.
If other analysts can't review
your mappings, try to make sure that
you're at least doing it the same
way as you go across different
reports and how you're using a given
technique.
So it can also be tempting and
once your experience for the tack,
you'll probably occasionally skip
steps and the mapping process,
you may be going straight to
identifying an applicable technique
or some technique.
You know, maybe you don't need to go
through a long process to identify
fishing, spearfishing attachment
in a report.
But it is important to remember
that this does increase your bias.
It's drawing from availability
bias, the techniques that you
have in your head that you're
already familiar with versus
the full range of techniques.
So it's probably something you're
eventually going to work up to
doing, but it wants
to always be done with a little bit
of caution.
So I'm going to get into one
of my favorite areas of cyber threat
intelligence, which is
biases in intelligence reporting.
I'm going to talk about this
specifically in terms of attack
map data. But these exist
all throughout cyber threat
intelligence.
The first thing about biases
is it's important for us to
recognize that they exist
and to understand some of what our
biases are in
cyber threat intelligence.
I'm going to get into two key types
of bias in
areas like the technique, examples
that are in attack.
And so these these biases exist
in the data that the attack team
puts out in attack groups
and software mappings as well as
work. You do yourself in mapping
attack.
And so the two types, I'm going to
get into our bias
introduced by us as consumers
and bias that's inherent in the
types of sources we use.
Understanding these biases is
the critical first step in
effectively leveraging the data.
So the first bias is in
the set of sources that you use.
So these are actually the
percentages for
reports that we have in
tax groups and software packages
and attacked up Miodrag.
The vast majority of
the material that we are able to
leverage is coming from security
vendors is mostly coming from
incident response.
In some cases, governments have put
out reports things like indictments
and in some cases there's high
quality press reporting that gets
into activities that adversaries
have done.
And it's not that there's anything
bad with this, but it's important to
understand the biases in the
specific sources that
are making up our intelligence.
And your set of sources is probably
going to look a little bit different
than this attack has some specific
constraints in using free and open
source threat intelligence
reporting.
Another set of biases that
we're going to have as
we go through data
is novelty and availability
bias.
And as the attack team, we're
absolutely
hit with novelty bias.
So somebody is putting out a report.
It's talking about
technique. We've seen a bunch
before, a group.
We've seen a bunch before.
So Fuzzy Duck is
using power shell again.
It might not be as interesting for
us and it might not come up to the
top of the queue as fast for us
adding it to attack, whereas
some brand new technique we have out
transmen of data manipulation,
very few actors do that.
We don't have a report in on Apte
Leet.
So actively using transmitted data
manipulation is going to be a lot
more novel to us and
we just need to be aware of these
biases.
And are they blinding us to
important activity?
As we wrap ourselves, we can have
that availability bias
attack as big as hundreds
of techniques, and there are going
to be a subset of techniques that
you remember in your head that you
know that you're familiar with.
You can immediately go and map to
and you're more likely to find those
techniques in reporting.
So it's important to recognize
if you found a certain frequency
of a technique to understand what
that may mean in terms of
how many times it occurs.
There are also biases in the sources
we use.
So to get into to
availability and visibility,
the people that are creating the
reports we're using have their own
availability bias.
They have behaviors they've seen
adversaries do before.
They understand as they're looking
at their data and are able
to go in and analyze
those behaviors a lot more
effectively.
Where is there is a wider range
of things that adversaries might
actually be doing.
Also, visibility, bias.
So a lot of the data that we're
talking about is coming from
incident response.
So it's likely that it's only types
of data that can be gathered after
an intrusion that are being included
in the reports.
So there are certain types of
behaviors that appear more
depending on the type of sensing and
the type of data that you're
actually able to use in your
reporting.
It's also victim novelty biases
in the reporting we use
victim bias is
that some victims are going to be
potentially more interesting
to be reported upon and
so more likely to generate a report.
So it could be some particular
industry, a really big name
company, an incident happening
to the other
way that who the victim is impacts
the reporting is that in a lot
of cases, companies are getting
permission from victims before
reporting on them, even if it
is anonymously.
So it can have a big impact
which type of industry, what kind of
reporting requirements they have
if the report even comes out at all
based on who the victim is.
And given that we're working off of
in a lot of cases, free
reports, there is some
novelty bias that we need to watch
out for, too.
And a lot of cases, threat
intelligence reports are coming out
of a marketing budget.
So there is some pressure where if
there's a group that's had frequent
past reporting,
maybe it's a little less likely.
We're going to see a report on this.
And we've seen this out in the wild
where groups like
Apte 10, there was no reporting
on them for a number of years, even
though people were seeing them out
in the wild, but they just weren't
doing anything particularly new in
industry.
Interesting.
Lo and behold, they break into a
bunch of service providers and
suddenly there's reporting out there
again, whereas it might
be more interesting to get a report
out on the new group on the block,
APTE one three
three eight instead of a lead.
So these aren't bad,
they exist, they're things that
we need to recognize and
there are some strategies we can
take for hedging these biases.
And the first is that step
five that we gave
collaborate.
If you're collaborating with others,
it can help mitigate especially your
own biases,
diversity of thought and diversity,
period on your teams makes for
stronger teams,
adjust and calibrate your data
sources, understand how your data
is potentially skewed
and adjust for that as you work with
it.
Try to work with as diverse a set of
sources as you can.
And oftentimes the absolute
best data is going to be that you
gather yourself where you've got
full access to all the information
around it and have a much
better idea of how the data is
shaded.
And finally, in working with all of
this, you were talking about gaps,
we're talking about places we might
not see, but we do
have is an opportunity to prioritize
the known.
Hopefully everything we're talking
about are things that we do know
as opposed to worrying about the
unknown.
It does mean that we may not be able
to say that, you know,
spearfishing attachment is more
popular than
a supply chain compromise,
but it does mean that we can say,
OK, we've seen both of these
existing out in the wild.
So we've gone over a couple of
things in here, talked about why
it's so important to work with
other analysts to collaboratively
assess attack mappings as well as
other threat intelligence.
And I've gotten into some key types
of bias that you're trying
to hedge as you do that
collaboration and other work
with threat intelligence.
So I've just gotten into the
second part of
this attack for CGI Journey,
getting into a number of steps to
help you map narrative data
to attack.
Next up, we're going to be going
into how you can apply that same
process to work with rawData.