[SOUND EFFECTS PLAYING]
Welcome to Intro to Web.
In this course, we're going to get into what a website is, how
web protocols work, and then we'll
dive into identifying some web directories.
So what is a website?
Really, they're just files on a server
that you're accessing, generally through a browser.
So basic three files you're likely to see, HTML, CSS,
and JavaScript, with HTML being your HyperText
Markup Language, which is just the basic format
of what's on that site.
Then for your presentation, you've
got a Cascading Style Sheet.
And then for functionality and anything additional,
you've got JavaScript.
And when you reach out with your browser,
you're going to pull these files using HTTP.
Likely, if someone is hosting locally on their own servers,
they'll be using Apache; NGINX; Tomcat;
or if it's a Windows-based server, IIS.
If you have off-premise hosting, which
is way more common nowadays in your cloud providers,
be it AWS, Azure, or Google Cloud platform,
there's also other hosting services,
like WordPress, Firebase, DigitalOcean, Cloudflare.
And they all provide places to host these files,
so they're accessible to all your users
without necessarily having to carry
the burden of owning that infrastructure yourself
or your organization.
With that, we use our browsers to connect to these servers
and pull these files utilizing HTTP.
That's the most common.
That's what we generally all use nowadays,
and we'll get into HTTPS in a little bit.
But the basics, HTTPS is just additional to HTTP.
You've got headers that go along with your requests
and your responses.
We'll talk about browsers and sessions,
because HTTP on its own does not maintain sessions
so we use something special.
So what is a header?
Headers hold information about what
you're requesting when you want the files from a server.
So there's certain methods, and we'll
dive into all of the methods.
Because remember, these are files being hosted,
so what directory are we going to start with?
Your host, what is the actual location of the server?
Now, we've, in information gathering,
gotten into domains and subdomains.
This is specific for each request.
And then another big thing that you're
going to get used to seeing in headers for HTTP
are user agent, which identifies what
"browser" you are using to make that request
because certain browsers require specific functionality.
And web designers, web developers
develop based on the platform that their users are
likely to use and create cross-functional things.
So based on your user agent string, whatever
you identify as your browser, they
provide what your browser is used to projecting to you.
Well, we get a lot out of that.
We can utilize that, and we can spoof it.
What you get back from the server
is going to be a response.
You send a request.
You get back a response.
No matter how you're interacting with those files,
it's not always reading.
We get into that with methods.
But either way, the client sends a request and the server
sends a response.
Big things in the headers for your responses
are your status codes.
We're going to get a lot out of that.
And then content type, there's a lot of information there.
And this is not a computer science course.
This is web app penetration testing.
I'm only teaching you what you need
to know here so that you can get started with web app
penetration testing as a whole.
So we request the client interaction with the server
by requesting things, and we do that through methods.
These are the methods that are available.
I'll tell you TRACK is the one that may or may not
be used by every browser out there, but GET is very common.
That's retrieving a file.
HEAD is just going to pull the headers.
POST posts like a form or a log-in.
That's going to be a post where you
send specific data to the server,
but it's going to still return you something specific.
There's PUT for putting files onto the server.
Yes, that is possible.
Or DELETE if you want to delete things off,
and we'll get into that in the lab.
There's also CONNECT, which may or may not be used.
Not super common.
OPTIONS allows you to know which of these methods
are available as options.
And then PATCH, again, may or may not
be available but to patch the software on the server.
Again, based off of our user agent,
we might get different sorts of information.
So just think about how we might be
able to use that as penetration testers.
What you get back is a response.
So the server sends resources.
And to let your browser know what it's receiving,
there's status codes, with 200 being OK,
like that was a good request.
Here's what you asked for.
So a 200 is an OK.
A 302 is a redirect.
The website that you're requesting actually
goes to another website, and the browsers and HTTP
work that out and already send a GET request usually
to that new page.
A 404, you might be familiar with that.
Resource not available or not found.
So what is a browser?
A browser just interprets and processes these files.
They take the HTML, which then connects
to your CSS and your JavaScript, and serves it up
for you in a fashion that you can work with.
There's other files.
There's PHP, which is just a software file that's
based on the backend.
It's just another way for it to send those files,
but we'll get into that with some examples in the future.
So then sessions.
I mentioned this a little bit before.
HTTP does not maintain sessions as a protocol in and of itself.
Instead, it utilizes cookies.
Cookies are codes that go alongside your requests.
And as long as that cookie stays there for as many times
as you're connected to the website,
then your session remains.
If you log in, then you're going to get a cookie.
If you add things to your cart, you're going to get a cookie.
There's also tracking cookies.
There's authentication cookies.
I talked about logging in.
There's different versions of logging in.
Maybe if you have an admin account,
then you get a different sort of authorization cookie.
Cookies can be very helpful.
But again, tracking cookies might be invasive.
And there are some governments that have said,
hey, yeah, tracking cookies of a certain type are invasive
and they're not allowed.
But in and of themselves, they're not a problem.
They're helpful.
Think about, as a pen tester, what could you
take advantage of with cookies?
Maybe not your session but maybe someone else's session.
That's a little more advanced than what
we're going to get into, but it's worth thinking about
and it's very important.
So HTTPS, this is very common.
It's no longer 2017.
HTTPS is what everyone is using.
It is now the standard.
It is HTTP secure, and it just wraps
your application-level data in an application-level encryption
of TLS or SSL.
And what it does is it takes your information.
Here's a couple examples.
This is a POST request and what it looks like over HTTP
and a POST request over HTTPS.
It's encrypted so that it looks like gobbledygook
to anyone listening, even if it's on the same machine.
If you have the keys, you can decrypt it.
But likely, anyone sniffing can't make out
what is going on there, so that is HTTPS.
We're going to jump into an environment now,
and I'll show you a little bit more
about these things that I mentioned in the slides.
Get my face out of the way.
We have demo.ine.local.
We're just going to ping it and it is up.
And it gives us an IP address of 192.35.202.3.
We're just going to navigate there.
demo.ine.local, it says bWAPP from OWASP.
Great little web app to identify some of these things.
Let's run an Nmap scan on that domain, demo.ine.local.
And we can see it's got port 80 open hosting on HTTP and MySQL,
and we could run that again.
And we'll just do some service enumeration on those, too.
We'll have that run.
But we can see, we've got the website here.
If you look at the website, it says /login.php.
I mentioned this a little bit before but .php is a file type
often used by Apache.
And it's a language in and of itself
utilized by the Apache service to then serve up
the proper files.
So when I say "files," let's take a look at the page source.
And here, we have the HTML code.
There's the head information with some metadata and links
to the style sheet, which is that CSS, stylesheet.css.
And we've got some scripts that are over here in html5.js.
These are the other files utilized and connected
to the HTML that is served up by this file login.php located
on the web server.
And again, not a computer science or a web
app or a web design course.
There's ways of going and learning
what "head" means and "body," headers and these lists
and tables.
Highly recommend taking a course specific to this
that we should have in our fundamentals area,
just so you can understand what all this means.
Because if you're going to do web app penetration testing,
you need to understand what it is you're looking at.
We can also Inspect Element.
And in here, we can look at storage.
And see, we've got a cookie for a PHP session ID
and it tells us where the path is and the domain.
And this is just setting things up.
So as we move around this website,
we've got some persistence, really.
So that as we continue to do stuff, it kind of makes sense.
And if we went into the Style Editor,
we could see what the CSS looks like.
And there's a console to interact with the JavaScript,
but here is your HTML.
And you can see what each element is related
to in the stylesheet, and you can often
see if there's any JavaScript connected to it.
But I'm not going to get in too much more of that.
I'd show you how it's a file that's served up on that server
and that we're looking at it.
Cookie Editor is awesome.
If you get Kali, you'll play with that.
So this right here is an HTTP page
because it's hosted on port 80 on HTTP service,
so it is not secure.
Meaning that, if we spin up Wireshark
and we listen in on eth1, connect here, let's to do this.
Bee, bug.
You can see, it's right there.
If we pop back over and search for HTTP,
actually let's do frame contains bee.
We have our login.php.
Let's follow this HTTP stream.
And right there, we have our login.php.
We did a POST request with these variables for the login
as bee and the password as bug.
And it included a little cookie that we didn't think about,
security level of 0.
And the form was submitted.
This is what we sent in.
And we had a cookie, our PHP session ID, to maintain.
And it came up with a redirect, a 302, which then automatically
sent to portal.php.
So you had login.php.
It went to portal.php.
I didn't request that.
It happened when we logged in.
And then that's served up a 200.
And we ended up with this HTML, which
is what we're looking at now.
So that's the basics of how these files go across
and how we interact with the browser.
We could also use a tool like curl.
And curl is just going to pull over that page,
and there wasn't anything for it to show.
Oh.
There wasn't anything for it to show
because curl doesn't automatically
follow the redirect.
I added verbosity to my curl request.
Curl does the same thing that a browser does,
which is very specific in your command line.
So I said "curl" and then this domain,
and so it sent a GET request to that host
and its user agent was curl.
And what we got back was a 302, which is a redirect.
That's all it learned, was that there was a redirect.
And it says that the location of the redirect is portal.php,
so that's kind of useful.
Now, let's take a look at demossl.ine.local,
and that one's up.
And let's run an Nmap on demossl.ine.local.
Typos happen.
And it's got both port 80 open and port 443,
and that same MySQL server.
So let's look at the ports on that one along with services.
That's really what we want, is service enumeration.
While that's coming up, let's go take a look.
I'm going to restart this one.
Continue without saving.
And we go to demossl.ine.local.
And from there, let's log in.
And we were able to log in.
In this case, we see HTTPS.
That's my login.
But connection's secured, and we have this certificate.
Here.
We'd see who it was issued by and the public key
and the fingerprints.
And this is how it sets up our encryption.
Knowing what these keys are, if we had set it up
and grabbed the private keys, then we could decrypt this.
But we have this redirect.
Because initially, it went to a null HTTP, got the 302,
and sent back servers at demossl--
ah, document is moved to HTTPS demossl.ine.local.
It automatically followed that.
You can see.
Pause TCP stream.
And then that launched into this.
Because right before that, it started encryption.
Looks like the only things not encrypted
in there, some strings.
But this is the best it tried to do
of taking that encrypted data and trying to pull
any sort of strings out of it.
And this is the difference between HTTP and HTTPS.
And we can see, it's still just running Apache.
It's the same exact version.
It's just configured and running on 443, configured with SSL.
And that's the difference there.
Now, let's talk a little bit about directories
showed in here.
Let's pull back up.
There we have portal.php.
Well, what other directories exist in here?
We're going to use dirb, because it is fast
and we'll just do demo.ine.local.
And what dirb is going to do is it's baked into Kali,
and it's going to do some directory busting for us.
Has generated words.
It used this word list.
User shared dirb with this common.
And that's the default for dirb, which, for most examples
here, are going to work fine.
And it found a lot of directories.
There's an admin directory.
There's apps.
There's fonts.
There's images.
And you might be thinking, well, what does that mean?
And that would be a great question.
So let's go to demo.ine.local and fonts.
That's a directory.
These are different fonts that are being used.
They'll be utilized by the CSS.
There's also images.
These are the images that are hosted on different pages,
but they're stored in the same server in an images directory.
We saw that there is an admin, and that takes us
to an admin portal.
This is a finding depending on what you can do with it.
All right.
There are credentials, bee, bug.
Those are still logged in.
There's the JavaScript, so you could see the files
of what is in this JavaScript.
Another great one is robots.txt.
Your different search engines out there
need to know what they should scan
and what they shouldn't scan to include.
Did I really just do that for this one?
In their results and what they should spider,
so robots.txt tells you.
The user agent GoodBot is nothing is disallowed.
BadBot, everything from root down is disallowed.
And then everything else, don't go through admin.
Don't spider admin or documents or images or passwords.
Passwords has me wanting to look at things,
and we've got passwords.
Well, that's not good.
Different login IDs and their passwords.
And looks like some of their information
to recover their passwords.
So that's why it's important to check robots.txt and see what's
not allowed, because what's not allowed
might be things that we want to look at as pen tester.
Should they not be available at all?
All right.
So those are web directories and websites and how they work.
I hope that that was informative and that you
learned a little bit.
If this is your first foray into web app pen testing,
then I hope this gets you a little taste.
We're going to now dive into some labs
and get into some more specifics, more
tools for directory busting, and some exploits.
So hang on for those.