Introduction – NLP through Machine Learning
Natural Language Processing, NLP in short is an area of machine learning focused on algorithms that can analyze human language. NLP has a huge number of applications since it enables intelligent solutions to our day to day problems. Applications of NLP include voice assistants (Siri/Alexa) , chat bots, question answer systems, automated email responses etc. All these models have the capability to understand intent in human natural language and respond accordingly. Personally I find this area of machine learning very exciting and promising.
NLP or Natural Language Processing is truly revolutionizing the area of Machine Learning that is focused on algorithms to analyze Human Language. Natural language processing machine learning is getting widely adopted in many daily use applications – Voice assistance, chat bots, question answer system, automated email responses, etc. If you are interested or looking for a partner to apply natural language processing to your business, please contact us to discuss your use case.
NLP before and after deep learning
Before the success of deep learning, NLP problems were addressed through features that looked at count of words (bag of words), or relative importance of individual words in the overall document (tfidf) relying primarily on the presence of words. This often resulted in challenges related to curse of dimensionality since the number of words in a document may exceed the number of examples and we had sparse data. Also tougher problems like machine translation were addressed through carefully hand crafted features.
The success of deep learning has bought a big change in how we address NLP problems now. One of the fundamental innovations that allows us to perform better at NLP tasks is the representation of words as vectors commonly called as word2vec. Word2vec represents each word as say a 100 dimensional vector and the training of word2vec allows us to associate meaning to words. Words that are similar to each other will have vectors that are closer together in the vector space. I think of this as giving our knowledge of language to the computer. You can read more about word2vec and how to train them on blog here.
Key Areas of NLP where you can apply Machine Learning
I have built many applications that solve practical NLP problems through use of machine learning. In this blog, I talk about a few of them:
Text Classification
Text Classification is the task of assigning a label to a piece of text which would be a single sentence or a document. Some common examples include sentiment classification, predicting user rating, detecting if an email is spam or not. Deep Learning has made it easy to use RNNs and CNNs for Text Classification.
I did a mini project using IMDB reviews to classify a review into either a positive or negative review. I used a Multi Layered RNN to classify reviews with an accuracy of 83%
For more detail, please see my Github link here.
Besides RNN, CNNs can also be used for text classification. This blog explains it beautifully. I built a CNN for classifying financial services customer complaints into different departments and presented this at Google NEXT Conference.
Text Summarization
Deep Learning has allowed us to to make progress on Abstractive Text Summarization. This is a sequence to sequence task where input and output sequences can be of different lengths. I have built a Pointer Generator Model to do summarizations. The model gets a Rouge Score of 0.39 which is state of the art for Abstractive Summarization. For more information, please check out my blog here. ”
Question Answer Model
NLP can also help find answers to question in a paragraph! SQUAD data set has made this possible. Infact some of the latest models are able to get to human level accuracy in answering questions from a context. Examples of context, question and answer are:
Context — Apollo ran from 1961 to 1972, and was supported by the two-man Gemini program which ran concurrently with it from 1962 to 1966. Gemini missions developed some of the space travel techniques that were necessary for the success of the Apollo missions. Apollo used Saturn family rockets as launch vehicles. Apollo/Saturn vehicles were also used for an Apollo Applications Program, which consisted of Skylab, a space station that supported three manned missions in 1973–74, and the Apollo–Soyuz Test Project, a joint Earth orbit mission with the Soviet Union in 1975.
Question — What space station supported three manned missions in 1973–1974?
Answer — Skylab
I built a ensemble model using BIDAF and RNET architectures that does very well on the SQUAD data set. I have explained these models in details and shared the full code online. If you are interested, read my blog here.
There are many more areas where Deep Learning is having astounding success in NLP. If you have a specific use case in mind, please contact me through our Contact Us section.