In the previous video, I showed you how to load PDF files into Lanchain documents.
However, your private unstructured data isn't limited to PDF format.
It can be found in various other formats, such as Office documents, Google Docs,
and more.
Next, I'll demonstrate how to load these
different document formats into Lanchain.
Let's change the loadDocument function so
that it accepts other formats as well.
The function takes the filename to load
as an argument.
I'm gonna check the extension of the file
and based on its extension, I'll load it using the specific Lanchain loader.
I'm importing OS.
I'm splitting the filename into name and extension.
So name, extension equals OS .path .splittext of file.
You can print name and extension if you want to see their values.
If the extension is .pdf, I'll use the existing code which loads PDF documents.
So if extension equals equals .pdf, and I'm loading the data using pypdf loader.
I'm indenting these three lines of code.
But if the extension is .docx, I'll use
the code that loads Office documents.
So, if extension equals equals .docx.
I'm importing the required library.
From Lanchain document loaders import
docx2txt loader.
This covers how to load Word documents
into a document format that we can use later in the application.
It is also necessary to install another Python module called docx2txt.
I'm installing the package.
pip install docx2txt -q
And I'm running the cell.
Ok, it was installed.
And I'm restarting the kernel to use the updated package.
And I'm running the cell that loads the environment variables.
Let's continue implementing the function.
I'm printing a message for the user.
It's the same message which says which file is loading.
And the loader equals docx2txtloader of file.
You'll add an elif branch for each
document format you want to support.
There are so many.
Take a look here.
When you are done with all the formats,
add an else clause and print a message for the user.
Like this.
else print document format is not supported.
And I'm returning none.
Outside if elif else, I'm loading the
data and returning it.
Let's test the function.
I'm running the cell.
And here in the running code section, I'm
going to load a docx file that contains the novel The Great Gatsby by F. Scott Fitzgerald.
It's in the files directory.
This file.
Data equals load document of files and The Great Gatsby dot docx.
This time data is a list with a single element and the content is in the page
underlying content attribute.
So print data of zero dot page content.
And I'm running it.
It has displayed the document contents.
If you take a look at Lanchain document loaders in the documentation, you'll
notice that in addition to transform loaders, there are also public and
proprietary data set or service loaders.
We'll take a break and in the next video,
I'll show you how to load the data from public or private services.
Bye Bye.