Sentiment analysis of the Hamas-Israel war on YouTube comments using deep learning Scientific Reports

TextBlob is a library that provides a simple API for handling text data with tasks such as part-of-speech data, noun phrase extraction, tokenization, classification, and more. In 2021 I and some colleagues published a research article on how to employ sentiment analysis on a applied scenario. In this article — presented at the Second ACM International Conference on AI in Finance (ICAIF’21) — we proposed an efficient way to incorporate market sentiment into a reinforcement learning architecture. The source code for the implementation of this architecture is available here, and a part of it’s overall design is displayed below. Josh Miramant is the CEO and founder of Blue Orange Digital, a top-ranked data science and machine learning agency with offices in New York City and Washington DC.

To minimize the risks of translation-induced biases or errors, meticulous translation quality evaluation becomes imperative in sentiment analysis. This evaluation entails employing multiple translation tools or engaging multiple human translators to cross-reference translations, thereby facilitating the identification of potential inconsistencies or discrepancies. Additionally, techniques such as back-translation can be employed, whereby the translated text is retranslated back into the original language and compared to the initial text to discern any disparities. ChatGPT App By undertaking rigorous quality assessment measures, the potential biases or errors introduced during the translation process can be effectively mitigated, enhancing the reliability and accuracy of sentiment analysis outcomes. Take into account news articles, media, blogs, online reviews, forums, and any other place where people might be talking about your brand. This helps you understand how customers, stakeholders, and the public perceive your brand and can help you identify trends, monitor competitors, and track brand reputation over time.

• Exclusive indicates content/data unique to MarketsandMarkets and not available with any competitors.
• These algorithms work together with NER, NNs and knowledge graphs to provide remarkably accurate results.
• Sprout Social’s Tagging feature is another prime example of how NLP enables AI marketing.
• Bidirectional encoder representations from rransformers (BERT) representation.
• A key challenge that faces transcription and NLP tools is the capability to actually understand the tone of the speaker with sentiment analysis.

Text components are represented by numerical vectors which may represent a character, word, paragraph, or the whole document. In addition to gated RNNs, Convolutional Neural Network (CNN) is another common DL architecture used for feature detection in different NLP tasks. For example, CNNs were applied for SA in deep and shallow models based on word and character features19. Moreover, hybrid architectures—that combine RNNs and CNNs—demonstrated the ability to consider the sequence components order and find out the context features in sentiment analysis20. These architectures stack layers of CNNs and gated RNNs in various arrangements such as CNN-LSTM, CNN-GRU, LSTM-CNN, GRU-CNN, CNN-Bi-LSTM, CNN-Bi-GRU, Bi-LSTM-CNN, and Bi-GRU-CNN. Convolutional layers help capture more abstracted semantic features from the input text and reduce dimensionality.

Algorithms

It is necessary to integrate several different strategies in order to create the best possible mixture. All models cannot integrate with deep learning techniques at their initial level because all of the procedures need to be revised. A comparative study was conducted applying multiple deep learning models based on word and character features37. Three CNN and five RNN networks were implemented and compared on thirteen reviews datasets. Although the thirteen datasets included reviews, the deep models performance varied according to the domain and the characteristics of the dataset. Based on word-level features Bi-LSTM, GRU, Bi-GRU, and the one layer CNN reached the highest performance on numerous review sets, respectively.

Without doing preprocessing of texts, ULMFiT achieved massively good F1-scores of 0.96, 0.78 on Malayalam and Tamil, and DistilmBERT model achieved 0.72 on Kannada15. In recent years, classification of sentiment analysis in text is proposed by many researchers using different models, such as identifying sentiments in code-mixed data9 using an auto-regressive XLNet model. Despite the fact that the Tamil-English mixed dataset has more samples, the model is better on the Malayalam-English dataset; this is due to greater noise in the Tamil-English dataset, which results in poor performance. These results can be improved further by training the model for additional epochs with text preprocessing steps that includes oversampling and undersampling of the minority and majority classes, respectively10. In addition, deep models based on a single architecture (LSTM, GRU, Bi-LSTM, and Bi-GRU) are also investigated. The datasets utilized to validate the applied architectures are a combined hybrid dataset and the Arabic book review corpus (BRAD).

A sequential model such as an RNN or an LSTM would be able to much better capture longer-term context and model this transitive sentiment. Sebastian Raschka gives a very concise explanation of how the logistic regression equates to a very simple, one-layer neural network in his blog post. The input features and their weights are fed into an activation function (a sigmoid for binary classification, or a softmax for multi-class). The output of the classifier is just the index of the sigmoid/softmax vector with the highest value as the class label. Note that VADER breaks down sentiment intensity scores into a positive, negative and neutral component, which are then normalized and squashed to be within the range [-1, 1] as a “compound” score.

Why Do You Need Sentiment Analysis and NLP in Social Media?

Read on to get a better understanding of how NLP works behind the scenes to surface actionable brand insights. Plus, see examples of how brands use NLP to optimize their social data to improve audience engagement and customer experience. The software uses NLP to determine whether the sentiment in combinations of words and phrases is positive, neutral or negative and applies a numerical sentiment score to each employee comment.

If you are looking to for an out-of-the-box sentiment analysis model, check out my previous article on how to perform sentiment analysis in python with just 3 lines of code. The demo program uses a neural network architecture that has an EmbeddingBag layer, which is explained shortly. The neural network model is trained using batches of three reviews at a time. After training, the model is evaluated and has 0.95 accuracy on the training data (19 of 20 reviews correctly predicted). In a non-demo scenario, you would also evaluate the model accuracy on a set of held-out test data to see how well the model performs on previously unseen reviews.

Using progressively more and more complex models, we were able to push up the accuracy and macro-average F1 scores to around 48%, which is not too bad!. In a future post, we’ll see how to further improve on these scores using a transformer model powered by transfer learning. In 2018, Zalando Research published a state-of-the-art deep learning sequence tagging NLP library called Flair. This quickly became a popular framework for classification tasks as well because of the fact that it allowed combining different kinds of word embeddings together to give the model even greater contextual awareness. You can foun additiona information about ai customer service and artificial intelligence and NLP. The original RNTN implemented in the Stanford paper [Socher et al.] obtained an accuracy of 45.7% on the full-sentence sentiment classification.

As it is well known, a sentence is made up of various parts of speech (POS), and each combination yields a different accuracy rate. The validation accuracy of various models is shown in Table 4 for various text classifiers. Among all Multi-channel CNN (Fast text) models with FastText, the classifier gives around 80% validation accuracy rate, followed by LSTM (BERT), RMDL (BERT), and RMDL (ELMo) models giving 78% validation accuracy rate. Table 4 shows the overall result of all the models that has been used, including accuracy, loss, validation accuracy, and validation loss. Dropout layer is added to the top of the Conv1D layer with the dropout value of 0.5; after that, max-pooling layer is added with the pooling size of 2; after that result is flattened and stored in the flat one layer. Similarly, channels 2 & 3 have the same sequence of layers applied with the same attribute values used in channel 1.

Ultimately, doing that for a total of 1633 (training + testing sets) sentences in the gold-standard dataset and you get the following results with ChatGPT API labels. The next parts of this series will explore deep learning approaches to building a sentiment classifier. Recall that linear classifiers tend to work well on very sparse datasets (like the one we have).

Furthermore, stemming and lemmatization are the last NLP techniques used on the dataset. The two approaches are used to reduce a derived or inflected word to its root, base, or stem form. The distinction between stemming and lemmatization is that lemmatization assures that the root word (also known as a lemma) is part of the language. Investing in the best NLP software can help your business streamline processes, gain insights from unstructured data, and improve customer experiences.

Let’s use this now to get the sentiment polarity and labels for each news article and aggregate the summary statistics per news category. Natural Language Processing (NLP) is all about leveraging tools, techniques and algorithms to process and understand natural language-based data, which is usually unstructured like text, speech and so on. In this series of articles, we will be looking at tried and tested strategies, techniques and workflows which can be leveraged by practitioners and data scientists to extract useful insights from text data. This article will be all about processing and understanding text data with tutorials and hands-on examples. As standard in these recent pre-training times, we fine-tuned a BERT model with our proposed data set.

Slang and colloquial languages exhibit considerable variations across regions and languages, rendering their accurate translation into a base language, such as English, challenging. For example, a Spanish review may contain numerous slang terms or colloquial expressions that non-fluent Spanish speakers may find challenging to comprehend. Similarly, a social media post in Arabic may employ slang or colloquial language unfamiliar to individuals who lack knowledge of language and culture. To accurately discern sentiments within text containing slang or colloquial language, specific techniques designed to handle such linguistic features are indispensable. Another approach involves leveraging machine learning techniques to train sentiment analysis models on substantial quantities of data from the target language. This method capitalizes on large-scale data availability to create robust and effective sentiment analysis models.

If working correctly, the metrics provided by sentiment analysis will help lead to sound decision making and uncovering meaning companies had never related to their processes. Entirely staying in the know about your brand doesn’t happen overnight, and business leaders need to take steps before achieving proper sentiment analysis. One more great choice for sentiment analysis is Polyglot, which is an open-source Python library used to perform a wide range of NLP operations. The library is based on Numpy and is incredibly fast while offering a large variety of dedicated commands.

Before determining employee sentiment, an organization must find a way to collect employee data. For this, we will build out a data frame of all the named entities and their types using the following code. Thus you can see it has identified two noun phrases (NP) and one verb phrase (VP) in the news article. We will leverage the conll2000 corpus for training our shallow parser model. This corpus is available in nltk with chunk annotations and we will be using around 10K records for training our model. Considering our previous example sentence “The brown fox is quick and he is jumping over the lazy dog”, if we were to annotate it using basic POS tags, it would look like the following figure.

In this article, I will show you how to use Natural Language Processing (NLP) and more specifically sentiment analysis to understand how people really feel about a subject. The top two entries are original data, and the one on the bottom is synthetic data. Instead, the Tf-Idf values are created by taking random values between the top two original data. As you can see, if the Tf-Idf values for both original data are 0, then synthetic data also has 0 for those features, such as “adore”, “cactus”, “cats”, because if two values are the same there are no random values between them. I specifically defined k_neighbors as 1 for this toy data, since there are only two entries of negative class, if SMOTE chooses one to copy, then only one other negative entry left as a neighbour.

Explore the top 20 industries influencing your market growth

In the next post, I will try different classifiers with SMOTE oversampled data. And we can also see that all the metrics fluctuate from fold to fold quite a lot. Now we can see that NearMiss-2 has eliminated the entry for the text “I like dogs”, which again makes sense because we also have a negative entry “I don’t like dogs”. Two entries are in different classes but they share two same tokens “like” and “dogs”. It seems like both the accuracy and F1 score got worse than random undersampling. In NearMiss-1, those points from majority class are retained whose mean distance to the k nearest points in minority class is lowest.

The demo program concludes by predicting the sentiment for a new review of, “Overall, I liked the film.” The prediction is in the form of two pseudo-probabilities with values [0.3766, 0.6234]. The first value at index [0] is the pseudo-probability of class negative, and the second value at [1] is the pseudo-probability of class positive. I also made this same chart using the TextBlob Naive Bayes and Pattern analyzers with worse results (see the Jupyter notebook on my Github for these charts). The Naive Bayes model was trained on movie reviews which must not translate well to the Harry Potter universe. The Pattern analyzer worked much better (almost as well as VADER); it is based on the Pattern library, a rule-based model very similar to VADER. Companies can use customer sentiment to alert service representatives when the customer is upset and enable them to reprioritize the issue and respond with empathy, as described in the customer service use case.

Sentiment analysis

Employing LSTM, GRU, Bi-LSTM, and Bi-GRU in the initial layers showed more boosted performance than using CNN in the initial layers. In addition, bi-directional LSTM and GRU registered slightly more enhanced performance than the one-directional LSTM and GRU. Combinations of word embedding and handcrafted features were investigated for sarcastic text categorization54.

Polyglot is often chosen for projects that involve languages not supported by spaCy. TextBlob returns polarity and subjectivity of a sentence, with a Polarity range of negative to positive. The library’s semantic labels help with analysis, including emoticons, exclamation marks, emojis, and more. Sentiment analysis can also be used for brand management, to help a company understand how segments of its customer base feel about its products, and to help it better target marketing messages directed at those customers. With customer support now including more web-based video calls, there is also an increasing amount of video training data starting to appear. This “bag of words” approach is an old-school way to perform sentiment analysis, says Hayley Sutherland, senior research analyst for conversational AI and intelligent knowledge discovery at IDC.

Named entity recognition (NER) identifies and classifies named entities (words or phrases) in text data. These named entities refer to people, brands, locations, dates, quantities and other predefined categories. NLP uses rule-based approaches and statistical models to perform complex language-related tasks in various industry applications. Predictive text on your smartphone or email, text summaries from ChatGPT and smart assistants like Alexa are all examples of NLP-powered applications.

Business rules related to this emotional state set the customer service agent up for the appropriate response. In this case, immediate upgrade of the support request to highest priority and prompts for a customer service representative to make immediate direct contact. Logistic regression predicts 1568 correctly identified negative comments in sentiment analysis and 2489 correctly identified positive comments in offensive language identification.

The obtained results demonstrate that both the translator and the sentiment analyzer models significantly impact the overall performance of the sentiment analysis task. It opens up new possibilities for sentiment analysis applications in various fields, including marketing, politics, and social media analysis. We have studied machine learning models using various word embedding approaches and combined our findings with natural language processing. During the analysis phase, the priority is predominantly on providing more detail about the operations performed on the dataset by BERT, Glove, Elmo, and Fast Text. An investigated was performed on wide range of combinations of NLP and deep learning strategies, as well as methodologies considered to be cutting-edge. In order to build the best possible mixture, it is necessary to integrate several different strategies.

Generally speaking, an enterprise business user will need a far more robust NLP solution than an academic researcher. NLTK is great for educators and researchers because it provides a broad range of NLP tools and access to a variety what is sentiment analysis in nlp of text corpora. Its free and open-source format and its rich community support make it a top pick for academic and research-oriented NLP tasks. This list will be used as labels for the model to predict each piece of text.

• In another context, the impact of morphological features on LSTM and CNN performance was tested by applying different preprocessing steps steps such as stop words removal, normalization, light stemming and root stemming41.
• Facebook’s AI Research (FAIR) lab has created FastText, and basically, it learns word embeddings and text classification.
• If you do not do that properly, you will suffer in the post-processing results phase.
• Continuous evaluation and fine-tuning of models are necessary to achieve reliable results.

I am assuming you are aware of the CRISP-DM model, which is typically an industry standard for executing any data science project. Typically, any NLP-based problem can be solved by a methodical workflow that has a sequence of steps. In this article, we will be working with text data from ChatGPT news articles on technology, sports and world news. I will be covering some basics on how to scrape and retrieve these news articles from their website in the next section. If you do not have access to a GPU, you are better off with iterating through the dataset using predict_proba.

This helped them keep a pulse on campus conversations to maintain brand health and ensure they never missed an opportunity to interact with their audience. Text summarization is an advanced NLP technique used to automatically condense information from large documents. NLP algorithms generate summaries by paraphrasing the content so it differs from the original text but contains all essential information.

This score seems to be more reliable because it encompasses the overall sentiment of this corpus. But we can see from the scores above that tweets that have been classified as Hate Speech are especially negative. It’s diverse and constantly evolving, therefore, machine learning must constantly restrain models based on new lexicons.

Human translation offers a more nuanced and precise rendition of the source text by considering contextual factors, idiomatic expressions, and cultural disparities that machine translation may overlook. However, it is essential to note that this approach can be resource-intensive in terms of time and cost. Nevertheless, its adoption can yield heightened accuracy, especially in specific applications that require meticulous linguistic analysis. One of the primary challenges encountered in foreign language sentiment analysis is accuracy in the translation process.

It’s well-suited for organizations that need advanced text analytics to enhance decision-making and gain a deeper understanding of customer behavior, market trends, and other important data insights. Classic sentiment analysis models explore positive or negative sentiment in a piece of text, which can be limiting when you want to explore more nuance, like emotions, in the text. I found that zero-shot classification can easily be used to produce similar results. The term “zero-shot” comes from the concept that a model can classify data with zero prior exposure to the labels it is asked to classify. This eliminates the need for a training dataset, which is often time-consuming and resource-intensive to create.