Deep Learning for Natural Language Processing

Course Description

Starting with the basics, this book teaches you how to choose from the various text pre-processing techniques and select the best model from the several neural network architectures for NLP issues.

Overview

Applying deep learning approaches to various NLP tasks can take your computational algorithms to a completely new level in terms of speed and accuracy. Deep Learning for Natural Language Processing starts off by highlighting the basic building blocks of the natural language processing domain. The course goes on to introduce the problems that you can solve using state-of-the-art neural network models. After this, delving into the various neural network architectures and their specific areas of application will help you to understand how to select the best model to suit your needs. As you advance through this deep learning course, you’ll study convolutional, recurrent, and recursive neural networks, in addition to covering long short-term memory networks (LSTM). Understanding these networks will help you to implement their models using Keras. In the later chapters, you will be able to develop a trigger word detection application using NLP techniques such as attention model and beam search.

By the end of this course, you will not only have sound knowledge of natural language processing but also be able to select the best text pre-processing and neural network models to solve a number of NLP issues.

After completing this course, you will be able to:

• Understand various pre-processing techniques for deep learning problems
• Build a vector representation of text using word2vec and GloVe
• Create a named entity recognizer and parts-of-speech tagger with Apache OpenNLP
• Build a machine translation model in Keras
• Develop a text generation application using LSTM
• Build a trigger word detection application using an attention model

Course Length

3 days

Scope

Deep Learning with NLP perfectly balances theory and exercises. Each module is designed to build on the learnings of the previous module. The course contains multiple activities that use real-life business scenarios for you to practice and apply your new skills in a highly relevant context.

Target Audience

If you’re an aspiring data scientist looking for an introduction to deep learning in the NLP domain, this is just the book for you. Strong working knowledge of Python, linear algebra, and machine learning is a must.

Technical Requirements

Hardware:

For the optimal student experience, we recommend the following hardware configuration:

• Processor: Intel Core i5 or equivalent
• Memory: 4 GB RAM
• Storage: 5 GB available space

Software:

We also recommend that you have the following software installed in advance:

• OS: Windows 7 SP1 64-bit, Windows 8.1 64-bit or Windows 10 64-bit, Linux (Ubuntu, Debian, Red Hat, or Suse), or the latest version of OS X
• Python (3.6.5 or later, preferably 3.7; available through https://www.python.org/downloads/release/python-371/ )
• Jupyter (go to https://jupyter.org/install and follow the instructions to install). Alternatively, you can use Anaconda to install Jupyter.
• Keras ( https://keras.io/#installation )
• Google Colab: It is a free Jupyter notebook environment and runs on cloud infrastructure. It is highly recommended as it requires no setup and has pre-installed popular Python packages and libraries ( https://colab.research.google.com/notebooks/welcome.ipynb )

Course Outline

Lesson 1: Introduction to Natural Language Processing

• Basics of Natural Language Processing & application areas.
• Introduction to popular text pre-processing techniques.
• Introduction to word2vec and Glove word embeddings.
• Sentiment classification.

Lesson 2: Applications of Natural Language Processing

• Introduction to Named Entity Recognition.
• Introduction to Parts of Speech Tagging.
• Using popular libraries to develop a Named Entity Recognizer

Lesson 3: Introduction to Neural Networks

• Introduction to Neural Networks.
• Basics of Gradient descent and backpropagation.
• What is Deep Learning.
• Introduction to Keras.
• Fundamentals of deploying a model as a service.

Lesson 4: Foundations of Convolutional Neural Networks

• Introduction to CNN.
• Understanding the architecture of a CNN.
• Application areas of a CNN.
• Implementation using Keras.

Lesson 5: Recurrent Neural Networks

• Introduction to RNN.
• Understanding the architecture of a RNN.
• Application areas of a RNN.
• Implementation using Keras.
• Vanishing Gradients with RNN.

Lesson 6: Gated Recurrent Units

• Introduction to GRU.
• Understanding the architecture of a GRU.
• Application areas.
• Implementation using Keras.

Lesson 7: Long Short Term Memory

• Introduction to LSTM.
• Understanding the architecture of an LSTM.
• Application areas.
• Implementation using Keras.

Lesson 8: State of the art in Natural Language Processing

• Attention Model & Beam search.
• End to End models for speech processing.
• Dynamic Neural Networks for question answering.

Lesson 9: A practical NLP project workflow in an organization

• Data acquisition (Free datasets, crowd-sourcing)
• Using cloud infrastructure to train deep learning NLP model (Google colab notebook)
• Writing a Flask framework server RestAPI to deploy a model
• Deploy the web service on cloud infrastructure (AWS ec2 instance, docker)
• Current promising techniques in NLP (BERT and others).