Before actually implementing these workflows,
there are a couple of fundamental concepts that you need to be aware of.
Interacting with some of the other internal or online services that may not have
native clients or libraries is mostly done via a REST API.
For example,
they can be used to retrieve data that can be used to further enrich the
log data that is parsed and analyzed within the script,
but also to send data to another service for reporting or future analysis.
Regarding the technical aspects of interacting with REST APIs,
there are two important ones.
One is the format of the data that is being sent or received,
and the second is the module for initializing connections and sending requests.
Interacting with REST APIs requires data being sent
and received in a serialized format.
JSON is a very popular and widely used format.
Python, on the other hand, has built‑in support via the JSON module,
and it's very easy to convert lists and
dictionaries to and from the JSON format.
To make use of the JSON module, this has to be imported.
To generate the string in a JSON format,
the json.loads function is used, and on the opposite end,
we have the json.dumps to decode the JSON data.
Moving forward,
connectivity between the script and the REST APIs can be implemented with the
standard libraries that are available in Python already.
One topic that I want to have your attention on is
around authentication details.
Please don't store any keys or auth tokens within the script and rely
on external config files or environment variables to store these if
key management systems are not available.
There are two popular modules available,
one is urllib and the second one is requests.
The requests module is quite easy to use in just three lines,
like in this example.
You can make a request and retrieve data in no time.