> For the complete documentation index, see [llms.txt](https://ibm-developer.gitbook.io/python-and-analytics/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ibm-developer.gitbook.io/python-and-analytics/natural-language-understanding-and-sentiment-analysis/natural-language-understanding.md).

# Using Watson Natural Language Understanding to analyze sentiment

Natural-language understanding (NLU) is a subtopic of natural-language processing in artificial intelligence that deals with machine reading comprehension.[1](https://en.wikipedia.org/wiki/Natural-language_understanding) In this lab, we'll use Watson Natural Language Understanding to extract keywords from a data set and analyze them for the sentiment that is expressed. We'll do this in a Jupyter notebook using Python APIs and then we'll utilize [Pandas](https://pandas.pydata.org/pandas-docs/stable/index.html), [Matplotlib](https://matplotlib.org/index.html), and [Seaborn](https://seaborn.pydata.org/index.html) to visualize this information to aid in extracting insights.

In this workshop, we will learn how to:

* Create and use and instance of Watson Natural Language Understanding
* Connect to Watson Natural Language Understanding using Python APIs
* Perform analysis on text to derive Keywords, Sentiment, and Emotion
* Depict our information graphicaly to better understand the data

If you have not already done so, make sure that you do the work for your [project setup](/python-and-analytics/project-setup/project-setup.md)

## Create a Watson NLU instance

* Click on the *Navigation Menu* in the upper-left corner of your [Cloud Pak for Data as a Service](https://dataplatform.cloud.ibm.com/) instance and then click `Services` -> `Services catalog`: 

!\[CPDaaS Navigation Menu Services catalog(../assets/images/nlu/nlu-cpdaas-service-catalog.png)

* Type *Natural Language Understanding* in the search bar (the smart search will narrow down the choices as you type) and then click on the tile for `Natural Language Understanding`:

![IBM Cloud Natural Language Understanding](https://3604692573-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MGiMaRFLA6YbdmTsmru%2Fsync%2F48eaafb10c2007b4b98b140b931e0092cf1835bf.png?generation=1602017039000539\&alt=media)

* Choose the free `Lite` plan (unless your organization has already provisioned 1 instance of the `Lite` plan, in which case your subsequent plans would need to be on a paid tier). Give the service a meaningful name and click `Create`.
* You will see your newly created NLU instance on the *Services instances* page. Click on the name to open the page for this NLU instance:

![Newly created NLU Service insance](https://3604692573-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MGiMaRFLA6YbdmTsmru%2Fsync%2F9e9e5d6c3f68ee5b9ffaf3bfdb850ef871a47900.png?generation=1612389265692179\&alt=media)

* Either click on the `Credentials` tab of click on `View service credentials` to navigate to the *Credentials* page:

![Service Credentials page](https://3604692573-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MGiMaRFLA6YbdmTsmru%2Fsync%2Fda2acef16598bee70dbb82d89bd57dbcc440c86e.png?generation=1612389264623604\&alt=media)

* Click on `Create Credentials`. Next, click on the "down arrow" to view the newly created credentials. In the image below, you will see that I have obscured both the *apikey* and the *url*. You will need these later, so either copy them to a text document, or leave this tab open:

![Credentials with apikey and url](https://3604692573-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MGiMaRFLA6YbdmTsmru%2Fsync%2Ff5de18c05f9af742396cd2ab8d65c7a3fd082279.png?generation=1612389267001798\&alt=media)

Note that you can access the [Getting started tutorial](https://cloud.ibm.com/docs/services/natural-language-understanding?topic=natural-language-understanding-getting-started#getting-started-tutorial) and the [API reference](https://cloud.ibm.com/apidocs/natural-language-understanding) from the NLU Service instance *Overview* tab. You may wish to look at these for a more in-depth investigation of Watson Natural Language Understanding.

## Demonstration of Watson NLU and Sentiment Analysis

* Visit the [Live Demo of Watson Natural Language Understanding](https://www.ibm.com/demos/live/natural-language-understanding/self-service/home). Click the `Analyze text` button.

![Natural Language Understanding live demo](https://3604692573-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MGiMaRFLA6YbdmTsmru%2Fsync%2F1b7a7e0b3bf60478f27f5b3e7d1406ebef029dda.png?generation=1602017040134854\&alt=media)

* Click on the tabs for `Extraction`, `Classification`, `Linguistics`, and the `Custom` Entities. Notice that there are sub-tabs for each that you can investigate.

![NLU demo detailed tabs](https://3604692573-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MGiMaRFLA6YbdmTsmru%2Fsync%2F4bbbeb0cf7e7238a2f95c1707fc3f5ede4eafb08.png?generation=1602017042074869\&alt=media)

* Click on the tab for `Classification` and the sub-tab `Emotion`. You can see how NLU extracts information about the emotional content of the text.

![NLU demo Classification Emotion](https://3604692573-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MGiMaRFLA6YbdmTsmru%2Fsync%2Fe07d22a845f728165e23a1427ac2ee993b28fadc.png?generation=1602017041034519\&alt=media)

Here you can see the confidence scores for each of the various target phrases that were extracted, broken down into "Sadness", "Joy", "Fear", "Disgust", and "Anger".

Now that we've seen how Natural Language Understanding and Sentiment Analysis work, let's use the [Python APIs](https://cloud.ibm.com/apidocs/natural-language-understanding?code=python) in a Jupyter notebook along with our graphing and visualization tools.

## Load and Run a Notebook

* Go to your project on [Cloud Pak for Data as a Service](https://dataplatform.ibm.com)

![Select projects](https://github.com/IBM/python-and-analytics/tree/d1321ab18e44f93280860bea7d936b429aa5a09a/docs/assets/images/setup/cpd-create-project.png)

* In your project, click `Add to project` and choose `Notebook`:

![Add notebook](https://3604692573-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MGiMaRFLA6YbdmTsmru%2Fsync%2F864007c492b88e870df2cc0c682861e8201b0ead.png?generation=1599579707929887\&alt=media)

* Choose New notebook `From URL`. Give your notebook a name and copy the URL `https://github.com/IBM/python-and-analytics/blob/master/notebooks/nlu-sentiment-analysis.ipynb`. Click `Create`:

![Notebook from URL](https://3604692573-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MGiMaRFLA6YbdmTsmru%2Fsync%2F676f17f86934ac509763717fc695ca42254adae0.png?generation=1599579706811230\&alt=media)

Spend some time looking through the sections of the notebook to get an overview. A notebook is composed of text (markdown or heading) cells and code cells. The markdown cells provide comments on what the code is designed to do.

You will run cells individually by highlighting each cell, then either click the `Run` button at the top of the notebook or hitting the keyboard short cut to run the cell (Shift + Enter but can vary based on platform). While the cell is running, an asterisk (`[*]`) will show up to the left of the cell. When that cell has finished executing a sequential number will show up (i.e. `[17]`).