Reviews are the most helpful feature to know about any product and to predict its sell using analysis of the past costumer’s reviews. Sentimental analysis is the best tool to analyse all reviews without reading each review one by one, whether customers are happy or not happy with the product. It is being used to analyse movie reviews, many other products and also used to analyse tweets to survey support to any group or organization. Basically, the sentimental analysis classifies reviews in different classes like a positive review or a negative review. It reads the emotions behind any sentences and based on that emotions classify them. There are many NLP tools available for this task. BERT is the state-of-the-art model for NLP task nowadays with much better accuracy and solution to many different NLP problems. So here we have tried this BERT model for the sentimental analysis task.

BERT Introduction

When we came to know about opensource BERT (Bidirectional Encoder Representations from Transformers) model developed by the Google researchers, we started implementing for many Natural Language Processing task and it’s giving State-of-the-art results. In any NLP task, the main task is to train the base language model with large and large datasets is a big task and needs too many resources. So we decided to use this open-source BERT model with 110M parameters.

Pre-trained BERT models can be further fine-tuned for different NLP tasks. We tried to fine-tune it for the task of the sentimental analysis. In this task, we tried to classify the sentence into two classes whether it has positive sentiments or negative sentiments. This analysis can be used in many places like to classify the online store reviews or movie reviews to predict the probability to be well-known of any product or movie. It also can be used to classify the political tweets of the people to predict the tendency to win of a political party.

Fine-Tuning of BERT for sentimental task:

TensorFlow Hub is a library containing the trained models of machine learning. We have used the BERT-base uncased pretrained model available at the TF Hub with 110M parameters, 12 layers, 768-hidden, and 12-heads.

To achieve the task of classification, we have fine-tuned the pre-trained model. In fine-tuning, we have removed the last layer of the model and added our new neural network layer that classifies the text in two classes.


The Base BERT model was pre-trained on Wikipedia and Book Corpus, a dataset containing +10,000 books of different genres. So it gives benefits of pre-training of model on a much larger dataset which improves the ultimate results.

The pre-trained model can be fine-tuned on the dataset of sentimental analysis resulting in substantial accuracy improvements compared to training on these datasets from scratch. In the fine tuning task we have used the dataset of the IMDb Large Movie Review hosted by the Stanford. This is binary classification dataset contains 25,000 movie reviews for training and 25,000 for testing.

Demo of BERT Based Sentimental Analysis

We started working on to make the demo available for all NLP passionate and new learners who are eager to check our trained model results.

Try our BERT Based Sentiment Analysis demo.

Give input sentences separated by newlines. Due to the big-sized model and limited CPU/RAM resources, it will take a few seconds. Kindly be patient. In the results, it will give you sentiments for input sentences.

Do check out with demo and do let us know your feedback on BERT based sentiment analysis of the text in the comment section.

Accuracy/Results and Further roadmap:

The validation accuracy we gained after training of the model on the IMDb sentimental analysis dataset is 89.29%.

The ability of the BERT based fine-tuned model is just too good but not up to standard. Sometimes it gives wrong sentiments for the given input sentences. We would like to further fine-tune to improve its results.

We will also try to fine-tune this model on different large sentimental analysis dataset available and will check if it improves the results.

Feel free to comment your doubts/questions. We would be glad to help you.

If you are looking for Chatbot Development or Natural Language Processing services then do contact us or send your requirement at We would be happy to offer our expert services.

Categories: Natural Language Processing NLP

Leave a Reply

Your email address will not be published.

You may use these <abbr title="HyperText Markup Language">HTML</abbr> tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>