Trending March 2024 # Stemming Vs Lemmatization In Nlp: Must # Suggested April 2024 # Top 4 Popular

You are reading the article Stemming Vs Lemmatization In Nlp: Must updated in March 2024 on the website Cattuongwedding.com. We hope that the information we have shared is helpful to you. If you find the content interesting and meaningful, please share it with your friends and continue to follow and support us for the latest updates. Suggested April 2024 Stemming Vs Lemmatization In Nlp: Must

Introduction

In the field of Language Processing i.e., NLP, Lemmatization and Stemming are Text Normalization techniques. These techniques are used to prepare words, text, and documents for further processing.

Languages such as English, Hindi consists of several words which are often derived from one another. Further, Inflected Language is a term used for a language that contains derived words. For instance, word “historical” is derived from the word “history” and hence is the derived word.

There is always a common root form for all inflected words. Further, degree of inflection varies from lower to higher depending on the language.

To sum up, root form of derived or inflected words are attained using Stemming and Lemmatization.

The package namely, nltk.stem is used to perform stemming via different classes. We import PorterStemmer from nltk.stem to perform the above task.

For instance, ran, runs, and running are derived from one word i.e., run, therefore the lemma of all three words is run. Lemmatization is used to get valid words as the actual word is returned.

WordNetLemmatizer is a library that is imported from nltk.stem which looks for lemmas of words from the WordNet Database.

Note: Before using the WordNet Lemmatizer, WordNet corpora has to be downloaded from NLTK downloader.

Lemmatization and Stemming, both are used to generate root form of derived (inflected) words. However, lemma is an actual language word, whereas stem may not be an actual word.

Lemmatization uses corpus for stop words and WordNet corpus to produce lemma. Moreover, parts-of-speech also had to be defined to obtain correct lemma.

So, how to decide when to use what! If speed is important, use stemming as lemmatization scan the entire corpus which is a time-consuming task. Secondly, whether stemmers or lemmatizers should be used depends on the application we are working. Finally, if language is important while building a language application, lemmatization is used which scans a corpus to match root forms.

This article was published as a part of the Data Science Blogathon.

What is Stemming in NLP?

It is the process of reducing infected words to their stem. For instance, in figure 1, stemming with replace words “history” and “historical” with “histori”. Similarly, for the words finally and final.

Stemming is the process of removing the last few characters of a given word, to obtain a shorter form, even if that form doesn’t have any meaning.

Figure 1 showing Stemming

Why we Need Stemming?

In NLP use cases such as sentiment analysis, spam classification, restaurant reviews etc., getting base word is important to know whether the word is positive or negative. Stemming is used to get that base word.

Code for Stemming Explained

This section will help you in stemming of paragraph using NLTK which can be used in various use cases such as sentiment analysis, etc.

So let’s get started:

Note: It is highly recommended to use google colab to run this code.

Import Libraries

Import libraries that will be required for stemming.

import nltk nltk.download('stopwords') nltk.download('punkt') from nltk.corpus import stopwords from chúng tôi import PorterStemmer Get the Input

The paragraph will be taken as input and used for stemming.

paragraph = """ I have three visions for India. In 3000 years of our history, people from all over the world have come and invaded us, captured our lands, conquered our minds. From Alexander onwards, the Greeks, the Turks, the Moguls, the Portuguese, the British, the French, the Dutch, all of them came and looted us, took over what was ours. Yet we have not done this to any other nation. We have not conquered anyone. We have not grabbed their land, their culture, their history and tried to enforce our way of life on them. """ Tokenization (step before stemming)

Before, stemming, tokenization is done so as to break text into chunks. In this case, paragraph to sentences for easy computation.

As can be seen from output paragraph is divided into sentences based on “.” .

Stemming

In the code given below, one sentence is taken at a time and word tokenization is applied i.e., converting sentence to words. After that, stopwords (such as the, and, etc) are ignored and stemming is applied on all other words. Finally, stem words are joined to make a sentence.

Note: Stopwords are the words that do not add any value to the sentence.

Python Code:



From the above output, we can see that stopwords such as have, for have been removed from sentence one. The word “visions” have been converted to “vision, “history” to “histori” by stemming.

What is Lemmatization in NLP?

The purpose of lemmatization is same as that of stemming but overcomes the drawbacks of stemming. In stemming, for some words, it may not give may not give meaningful representation such as “Histori”. Here, lemmatization comes into picture as it gives meaningful word.

Lemmatization takes more time as compared to stemming because it finds meaningful word/ representation. Stemming just needs to get a base word and therefore takes less time.

Stemming has its application in Sentiment Analysis while Lemmatization has its application in Chatbots, human-answering.

Code for Lemmatization Explained

On similar lines of stemming, we will import libraries get input for lemmatization.

Import Libraries import nltk nltk.download('stopwords') nltk.download('punkt') nltk.download('wordnet') from chúng tôi import WordNetLemmatizer from nltk.corpus import stopwords Get the Input paragraph = """I have three visions for India. In 3000 years of our history, people from all over the world have come and invaded us, captured our lands, conquered our minds. From Alexander onwards, the Greeks, the Turks, the Moguls, the Portuguese, the British, the French, the Dutch, all of them came and looted us, took over what was ours. Yet we have not done this to any other nation. We have not conquered anyone. We have not grabbed their land, their culture, their history and tried to enforce our way of life on them. """ Tokenization (step before stemming) sentences = nltk.sent_tokenize(paragraph) print(sentences) Output: Lemmatization

The difference between stemming and lemmatization comes in this step where WordNetLemmatizer() is used instead of PorterStemmer(). Rest of steps are the same.

lemmatizer = WordNetLemmatizer() # Lemmatization for i in range(len(sentences)): words = nltk.word_tokenize(sentences[i]) words = [lemmatizer.lemmatize(word) for word in words if word not in set(stopwords.words('english'))] sentences[i] = ' '.join(words) Get the Output print(sentences) Output:

In above output, it can be noticed that although word “visions” have been converted to “vision” but word “history” remained “history” unlike stemming and thus retained its meaning.

Stemming vs Lemmatization

StemmingLemmatizationStemming is a process that stems or removes last few characters from a word, often leading to incorrect meanings and spelling.Lemmatization considers the context and converts the word to its meaningful base form, which is called Lemma.

For instance, stemming the word ‘Caring‘ would return ‘Car‘.

For instance, lemmatizing the word ‘Caring‘ would return ‘Care‘.Stemming is used in case of large dataset where performance is an issue.Lemmatization is computationally expensive since it involves look-up tables and what not.

Conclusion

One thing to note is that a lot of knowledge and understanding about the structure of language is required for lemmatization. Hence, in any new language, the creation of stemmer is easier in comparison to lemmatization algorithm.

The above points show that stemming should be used if speed is important since lemmatizers scan a corpus which is a time-consuming task. Further, the choice between lemmatizers and stemmers also depends on the problem you are working on.

Frequently Asked Questions

Q1. Which is better lemmatization or stemming?

A. The choice depends on the specific use case. Lemmatization produces a linguistically valid word while stemming is faster but may generate non-words.

Q2. Do you do both stemming and lemmatization?

A. As an AI language model, I can perform both stemming and lemmatization based on the task’s requirements or context.

Q3. Why is stemming faster than lemmatization?

A. Stemming chops off word endings without considering linguistic context, making it computationally faster. Lemmatization analyzes word forms to determine the base or dictionary form, which takes more processing time.

Q4. What is the application of stemming and lemmatization?

A. Stemming and lemmatization are used in natural language processing tasks such as information retrieval, text mining, sentiment analysis, and search engines to reduce words to their base or root forms for better analysis and understanding.

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.

Related

You're reading Stemming Vs Lemmatization In Nlp: Must

Top 30 Nlp Use Cases In 2023: Comprehensive Guide

Natural language processing (NLP) is a subfield of AI and linguistics which enables computers to understand, interpret and manipulate human language. 

Although machines face challenges in understanding human language, the global NLP market was estimated at ~$5B in 2023 and is expected to reach ~$43B by 2025. And this exponential growth can mostly be attributed to the vast use cases of NLP in every industry.

You may be familiar with many day-to-day NLP applications such as autocorrection, translation, or chatbots. However, NLP has numerous impactful applications that business leaders are not aware of. Therefore, we compiled a comprehensive list of NLP use cases and applications and categorized them according to relevant industries and business functions:

General applications

1. Translation

One of the top use cases of natural language processing is translation. The first NLP-based translation machine was presented in the 1950s by Georgetown and IBM, which was able to automatically translate 60 Russian sentences to English. Today, translation applications leverage NLP and machine learning to understand and produce an accurate translation of global languages in both text and voice formats.

2. Autocorrect

NLP is used to identify a misspelled word by cross-matching it to a set of relevant words in the language dictionary used as a training set. The misspelled word is then fed to a machine learning algorithm that calculates the word’s deviation from the correct one in the training set. It then adds, removes, or replaces letters from the word, and matches it to a word candidate which fits the overall meaning of a sentence.

3. Autocomplete

Autocomplete, or sentence completion, combines NLP with certain machine learning algorithms (e.g. Supervised learning, Recurrent neural networks (RNN), or Latent semantic analysis (LSA)) in order to predict the likelihood of using a following word or sentence to complete the meaning.

4. Conversational AI

Conversational AI is the technology that enables automatic conversation between computers and humans. It is the heart of chatbots and virtual assistants like Siri or Alexa. Conversational AI applications rely on NLP and intent recognition to understand user queries, dig in their training data, and generate a relevant response.

Chatbots have numerous applications in different industries as they facilitate conversations with customers and automate various rule-based tasks, such as answering FAQs or making hotel reservations.

For instance, Haptik produced a virtual assistant for Tata Mutual Fund to enhance customer retention and reduce call center workload. Initiative augmented the workforce of Tata by allowing employees to focus solely on urgent customer issues, by cutting call center enquiries by approximately 70%.

You can request a demo to see Haptik’s conversational AI solutions in action.

5. Automated speech/voice recognition

Voice recognition, also known as automatic speech recognition (ASR) and speech to text (STT), is a type of software that converts human speech from its analog form (acoustic sound waves) to a digital form that can be recognized by machines. ASR works by:

Splitting the audio of a speech recording into individual sounds (tokens),

Analyzing each sound,

Using algorithms (NLP, deep learning, Hidden Markov Model, N-grams) to find the most probable word fit in that language,

Converting the sounds into text.

Today, smartphones integrate speech recognition with their systems to conduct voice search (e.g. Siri) or provide more accessibility around texting. 

Source: Lekta

6. Automatic text summarization

Automatic text summarization is the process of shortening long texts or paragraphs and then generating a concise summary that passes the intended message. There are 2 main methods to summarize texts:

Cleaning the text from filling words

Sampling the text into shorter sentences (tokens)

Creating a similarity matrix that represents relations between different tokens

Calculating sentence ranks based on semantic similarity

Selecting sentences with top ranks in order to generate the summary (either extractive or abstractive)

7. Language models

Language models are AI models which rely on NLP and deep learning to generate human-like text and speech as an output. Language models are used for machine translation, part-of-speech (PoS) tagging, optical character recognition (OCR), handwriting recognition, etc.

Some of the famous language models are GPT transformers which were developed by OpenAI, and LaMDA by Google. These models were trained on large datasets crawled from the internet and web sources in order to automate tasks that require language understanding and technical sophistication. For instance, GPT-3 has been shown to produce lines of codes based on human instructions.

For more in-depth knowledge on sentiment analysis data collection, feel free to download our whitepaper:

Retail & e-commerce use cases

8. Customer service chatbot

A 2023 survey revealed that 65% of decision-makers in customer service believe that a chatbot can understand the customer’s context, and 52% said that chatbots can automate actions based on customer responses. Chatbots in customer service can:

For instance second hand car dealer Cars24, reduced its call center cost 75% by automating FAQs with a chatbot that deployed on WhatsApp and mobile app of the company.

To explore more use cases, feel free to read our in-depth article about chatbot use cases in customer service.

9. In-store bot

Several retail shops use NLP-based virtual assistants in their stores to guide customers in their shopping journey. A virtual assistant can be in the form of a mobile application which the customer uses to navigate the store or a touch screen in the store which can communicate with customers via voice or text. In-store bots act as shopping assistants, suggest products to customers, help customers locate the desired product, and provide information about upcoming sales or promotions.

10. Market intelligence

11. Semantic based search

Semantic search refers to a search method that aims to not only find keywords but understand the context of the search query and suggest fitting responses. Many online retail and e-commerce websites rely on NLP-powered semantic search engines to leverage long-tail search strings (e.g. women white pants size 38), understand the shopper’s intent, and improve the visibility of numerous products. Retailers claim that on average, e-commerce sites with a semantic search bar experience a mere 2% cart abandonment rate, compared to the 40% rate on sites with non-semantic search. 

Read our article on the Top 10 eCommerce Technologies with Applications & Examples to find out more about the eCommerce technologies that can help your business to compete with industry giants.

Healthcare use cases

12. Dictation

To document clinical procedures and results, physicians dictate the processes to a voice recorder or a medical stenographer to be transcribed later to texts and input to the EMR and EHR systems. NLP can be used to analyze the voice records and convert them to text, in order to be fed to EMRs and patients’ records.

13. Clinical documentation

14. Clinical trial matching

NLP can be used to interpret the description of clinical trials, and check unstructured doctors’ notes and pathology reports, in order to recognize individuals who would be eligible to participate in a given clinical trial. The algorithm used to develop such an NLP model would use medical records and research papers as training data in order to be able to recognize medical terminology and synonyms, interpret the general context of a trial, generate a list of criteria for trial eligibility, and evaluate participants’ applications accordingly.

A team at Columbia University developed an open-source tool called DQueST which can read trials on chúng tôi and then generates plain-English questions such as “What is your BMI?” to assess users’ eligibility. An initial evaluation revealed that after 50 questions, the tool could filter out 60–80% of trials that the user was not eligible for, with an accuracy of a little more than 60%.

15. Computational phenotyping

Phenotyping is the process of analyzing a patient’s physical or biochemical characteristics (phenotype) by relying on only genetic data from DNA sequencing or genotyping. Computational phenotyping uses structured data (EHR, diagnoses, medication prescriptions) and unstructured data (physicians vocal records which summarize patients’ medical history, immunizations, allergies, radiology images, and laboratory test results, as well as progress notes and discharge reports). Computational phenotyping enables patient diagnosis categorization, novel phenotype discovery, clinical trial screening, pharmacogenomics, drug-drug interaction (DDI), etc.

In this case, NLP is used for keyword search in rule-based systems which search for specific keywords (e.g. pneumonia in the right lower lobe) through the unstructured data, filter the noise, check for abbreviations or synonyms, and match the keyword to an underlying event defined previously by rules.

16. Computer assisted coding (CAC)

Computer Assisted Coding (CAC) tools are a type of software that screens medical documentations and produces medical codes for specific phrases and terminologies within the document. NLP-based CACs screen can analyze and interpret unstructured healthcare data to extract features (e.g. medical facts) that support the codes assigned.

17. Clinical diagnosis

NLP is used to build medical models which can recognize disease criteria based on standard clinical terminology and medical word usage. IBM Waston, a cognitive NLP solution, has been used in MD Anderson Cancer Center to analyze patients’ EHR documents and suggest treatment recommendations, and had 90% accuracy. However, Watson faced a challenge when deciphering physicians’ handwriting, and generated incorrect responses due to shorthand misinterpretations. According to project leaders, Watson could not reliably distinguish the acronym for Acute Lymphoblastic Leukemia “ALL” from physician’s shorthand for allergy “ALL”.

18. Virtual therapists

Virtual therapists (therapist chatbots) are an application of conversational AI in healthcare. NLP is used to train the algorithm on mental health diseases and evidence-based guidelines, in order to deliver cognitive behavioral therapy (CBT) for patients with depression, post-traumatic stress disorder (PTSD), and anxiety. In addition, virtual therapists can be used to converse with autistic patients to improve their social skills and job interview skills. For example, Woebot, which we listed among successful chatbots, provides CBT, mindfulness, and Dialectical Behavior Therapy (CBT).

Banking use cases

19. Stock prices prediction

NLP is used in combination with KNN classification algorithms to assess real-time web-based financial news, in order to facilitate ‘news-based trading’, where analysts seek to isolate financial news that affects stock prices and market activity. To extract real-time web data, analysts can rely on:

To learn how web scraping is used in finance, read In-Depth Guide to Web Scraping for Finance.

Sponsored

Bright Data’s Data Collector is a web scraping tool that targets websites, extracts financial data in real-time, and delivers it to end users in the designated format.

20. Credit scoring

Credit scoring is a statistical analysis performed by lenders, banks, and financial institutions to determine the creditworthiness of an individual or a business.

NLP can assist in credit scoring by extracting relevant data from unstructured documents such as loan documentations, income, investments, expenses, etc. and feed it to credit scoring software to determine the credit score.

In addition, modern credit scoring software utilize NLP to extract information from personal profiles (e.g. social media accounts, mobile applications) and utilize machine learning algorithms to weigh these features and assess creditworthiness.

Conversational banking can also help credit scoring where conversational AI tools analyze answers of customers to specific questions regarding their risk attitudes.

Sponsored:

Insurance use cases

21. Insurance claims management

NLP can be used in combination with OCR to analyze insurance claims. For example, IBM Watson has been used to comb through structured and unstructured text data in order to detect the right information to process insurance claims, and feed it to an ML algorithm which labels the data according to the sections of the claim application form, and by the terminology that commonly is filled into it.

Finance department use cases

22. Financial reporting

NLP can be combined with machine learning algorithms to identify significant data in unstructured financial statements, invoices, or payment documentations, extract it, and feed it to an automation solution, such as an RPA bot utilized for reporting in order to generate financial reports.

23. Financial auditing

NLP enables the automation of financial auditing by:

Screening financial documents of an organization

Classifying financial statement content

And identifying document similarities and differences

In turn, this enables the detection of deviations and anomalies in financial statements.

24. Fraud detection

NLP can be combined with ML and predictive analytics to detect fraud and misinterpreted information from unstructured financial documents. For instance, a study revealed that NLP linguistic models were able to detect deceptive emails, which were identified by a “reduced frequency of first-person pronouns and exclusive words, and elevated frequency of negative emotion words and action verbs”. The researchers used an SVM classifier algorithm to analyze linguistic features of annual reports, including voice, active versus passive tone, and readability, detecting an association between these features and fraudulent financial statements.

HR use cases

25. Resume evaluation

NLP can be used in combination with classification machine learning algorithms to screen candidates’ resumes, extract relevant keywords (education, skills, previous roles), and classify candidates based on their profile match to a certain position in an organization. Additionally, NLP can be used to summarize resumes of candidates who match specific roles in order to help recruiters skim through resumes faster and focus on specific requirements of the job.

26. Recruiting chatbot

Recruiting chatbots, also known as hiring assistants, are used to automate the communication between recruiters and candidates. Recruiting chatbots use NLP for:

Screening candidate resumes,

Scheduling interviews,

Answer candidates’ questions about the position,

Build candidate profiles,

Facilitating candidate onboarding.

27. Interview assessment

Many large enterprises, especially during the COVID-19 pandemic, are using interviewing platforms to conduct interviews with candidates. These platforms enable candidates to record videos, answer questions about the job, and upload files such as certificates or reference letters. NLP is particularly useful for interview platforms to analyze candidate sentiment, screen uploaded documentations, check for references, detect specific keywords which can reflect positive or negative behavior during the interview, as well as transcribe the video and summarize it for archiving purposes.

28. Employee sentiment analysis

Feel free to read our article on HR technology trends to learn more about other technologies that shape the future of HR management.

Cybersecurity use cases

29. Spam detection

NLP models can be used for text classification in order to detect spam-related words, sentences, and sentiment in emails, text messages, and social media messaging applications. Spam detection NLP models typically follow these steps:

Data cleaning and preprocessing: removing filling and stop words.

Tokenization: sampling text into smaller sentences and paragraphs.

Part-of-speech (PoS) tagging: tagging a word in a sentence or paragraph to its corresponding part of a speech tag, based on its context and definition.

The processed data will be fed to a classification algorithm (e.g. decision tree, KNN, random forest) in order to classify the data into spam or ham (i.e. non-spam email).

Source: Machine learning for email spam filtering: review, approaches and open research problems by Dada et al.

30. Data exfiltration prevention

Data exfiltration is a security breach that involves unauthorized data copying or transfer from one device to another. To exfiltrate data, attackers use cybersecurity techniques such as domain name system (DNS) tunneling (i.e. DNS queries which reflect a demand for information sent from a user’s computer (DNS client) to a DNS server) and phishing emails which lead users to provide hackers with personal information. NLP can be used to detect DNS queries, malicious language, and text anomalies in order to detect malware and prevent data exfiltration.

For more on NLP

To explore what natural language processing is, and what are its products, feel free to read our in-depth articles:

If you believe your business will benefit from a conversational AI solution, scroll down our data-driven lists of:

Additionally, you may download our whitepaper to get the most latest information about conversational AI:

And we can guide you through the process:

Links to Haptik on this page are sponsored.

This article was drafted by former AIMultiple industry analyst Alamira Jouman Hajjar.

Cem regularly speaks at international technology conferences. He graduated from Bogazici University as a computer engineer and holds an MBA from Columbia Business School.

YOUR EMAIL ADDRESS WILL NOT BE PUBLISHED. REQUIRED FIELDS ARE MARKED

*

0 Comments

Comment

Most Frequently Asked Nlp Interview Questions

This article was published as a part of the Data Science Blogathon.

Introduction

Natural language processing (NLP) is the branch of computer science and, more specifically, the domain of artificial intelligence (AI) that focuses on providing computers the ability to understand written and spoken language in a way similar to that of humans.

Combining computational linguistics (rule-based modeling of human language) with statistical, machine learning, and deep learning models is natural language processing (NLP). Together, these technologies enable computers to ‘understand’ the whole meaning of human language in the form of text or speech data, including the speaker’s or writer’s purpose and emotion.

NLP is the driving force behind computer systems that translate text from one language to another, respond to spoken commands, and swiftly summarise massive amounts of information—even in real-time. There is a strong probability that you have engaged with NLP through voice-activated GPS systems, digital assistants, speech-to-text dictation software, customer service chatbots, and other consumer conveniences. NLP plays an increasing role too in corporate solutions that optimize business operations, boost staff productivity, and simplify mission-critical business procedures.

What is the importance of natural language processing?

Businesses use the massive volume of unstructured, text-heavy data and require a method for processing it efficiently. Most of the data produced online and saved in databases are natural human languages. Until recently, organizations were unable to inspect this data efficiently. Herein lies the utility of natural language processing.

NLP Interview Questions

1. What do you mean by NLTK?

NLTK, which stands for Natural Language Toolkit, is a Python library. We use NLTK to process spoken language data. NLTK facilitates the application of techniques like parsing, tokenization, lemmatization, and stemming from comprehending natural languages. It aids in text categorization, parsing linguistic structure, document analysis, etc.

2. What is Parsing in regards to NLP?

In natural language processing, parsing refers to a machine’s understanding of a sentence’s grammatical structure. Parsing enables a device to understand the meaning of a word in a sentence along with the grouping of words, phrases, nouns, subjects, and objects. Parsing facilitates the analysis of a text or document to uncover valuable information.

3. What do you mean by Syntactic Analysis?

Syntactic analysis is a method used to derive meaning from sentences. A machine can examine and comprehend the order of words in a phrase through syntactic analysis. NLP makes use of the grammar rules of a language to aid in the syntactic analysis of the combination and order of words in texts.

Sentence: the dog saw a man in the park

Source: nltk.org

4. In NLP, what is Pragmatic Ambiguity?

Pragmatic ambiguity refers to words with several meanings whose usage in any given sentence is context-dependent. The same language may have several meanings due to pragmatic ambiguity. Most phrases we encounter contain words with several meanings, leaving them open to interpretation. This varied interpretation results in ambiguity and is referred to in NLP as Pragmatic Ambiguity.

5. What does Stemming imply in NLP?

Stemming is the process of eliminating suffixes from words to get their root form. It is comparable to chopping a tree’s branches into its trunk. For instance, the stem of eating, eats, and eaten is eat. Search engines index the words using stemming. Stemming is crucial for natural language comprehension (NLU) and natural language processing (NLP).

6. What do you mean by POS tagging?

Parts of speech tagging, often known as POS tagging, is the process of detecting individual words in a document and classifying them based on their context as parts of speech. POS tagging is also referred to as grammatical tagging since it requires understanding grammatical structures and identifying the corresponding chúng tôi tagging is a complex approach since the same word can have several meanings depending on context. For the same reason, the same basic approach employed for word mapping is unsuccessful for POS tagging.

7. What does Lemmatization imply in NLP?

Lemmatization is mapping a word’s different forms to its root (also known as the “lemma”). Although this may look similar to the definition of stemming, it is distinct. For instance, after stemming, the word “better” remains unchanged. Upon lemmatization, however, this should become “excellent.” Lemmatization requires a deeper understanding of language. Modeling and designing effective lemmatizers is still an open question in NLP research.

8. What does Text Normalization mean in NLP?

9. What does TF-IDF mean in Natural language processing?

TF-IDF, also known as Term Frequency-Inverse Document Frequency, is a method for determining the significance of a word relative to other terms in a corpus. It is a typical metric for information retrieval (IR) and summarization scoring. TF-IDF translates words to vectors and adds semantic information, resulting in weighted uncommon words that may be utilized in several NLP applications.

10. What is NES?

Named entity recognition, or NER, is finding entities in a written document that are more informative and have their context. These frequently signify locations, individuals, and organizations. Even though these items appear to be proper nouns, the NER method identifies much more than simply the nouns. In actuality, NER entails entity chunking or extraction, in which entities are split to classify them under several predetermined classifications. This process helps extract information further.

Conclusion

NLP is an area of computer science and AI (AI)

In NLP, parsing is a machine’s knowledge of a sentence’s grammar.

When you remove a word’s suffix, you return it to its original form.

TF-IDF stands for Term Frequency-Inverse Document Frequency

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion. 

Related

11 Best Apple Arcade Games You Must Play In 2023

Did you sign up for Apple Arcade and are now looking for some cool games that you can play to recharge yourself? Read on to find the best Apple Arcade games you should play if you buy a subscription.

You might love playing the best iOS RPG games or the best puzzle games on iPhone or iPad. But two things bug you the most during exciting and challenging gameplay.

The first one is endless in-app purchases that aim to empty your pocket. And the second one is third-party app ad videos, which mostly show up when you play the free version of game apps.

Yes, you heard it right! Now you can play top and trending App Store games unlimited times via an Apple Arcade subscription.

Here are the best apps in the Apple Arcade subscription package:

What Is Apple Arcade?

Apple Arcade is a subscription plan for App Store. For $4.99 per month, you can play more than 200 premium games on App Store. You can also share the subscription with up to five family members if you’ve got the Family Sharing.

You can also enjoy the one-month free trial if you’re unsure about subscribing. Additionally, Apple is offering a three-month free subscription with eligible Apple Devices.

If you’ve bought an Apple device recently, you might’ve won a free subscription too. Check your eligibility on the Apple Arcade website.

Moreover, these game apps are not just for iPhone and iPad devices. You can also play some of these games on Mac and Apple TV.

1. NBA 2K23 Arcade Edition

NBA 2K23 Arcade Edition is an exclusive game app from NBA 2K franchise for the Apple Arcade platform. The game is compatible with Mac, iPhone, iPad, and Apple TV. Therefore, you can sync the gameplay on multiple devices.

Start the game from your iPhone when commuting, resume it on your iPad at school or home, and play on big-screen Apple TV on weekends without fuss.

The game app features 30 NBA players who are “Greatest of All-Time” from the recently ended 2023–22 NBA season or the 76th NBA Season. Some of the star players against whom you can play in the game are Luka Dončić, Devin Booker, Kevin Durant, and more.

That’s not all! You can also play against legendary players like Shaquille O’Neal, Kareem Abdul-Jabbar, and Michael Jordan.

2. Angry Birds Reloaded

Angry Birds Reloaded is yet another big attraction of Apple Arcade subscription. As a premium treatment for its subscribers, Apple will offer Daft Piggies as an exclusive episode. This enhanced episode will contain 30 new exciting levels.

Its notable features are:

Power Up helps you to boost your birds’ strength so that they can destroy pig’s towers.

Eagles are the new additions in Angry Birds Reloaded. These creatures will try to wreak havoc on your island. You must muster your birds to defeat the Eagles.

There is a local leaderboard to track your accomplishments.

3. Sonic Racing

Looking for something different in racing games than grand theft auto? Try Sonic Racing with Sonic the Hedgehog. And there are 14 more charming characters like Miles “Tails” Prower, Shadow the Hedgehog, E-123 Omega, Silver the Hedgehog, Amy Rose, and more.

With this game, you get an immersive experience of the Sonic universe where characters team up online, play against opposite teams, set traps to defeat contenders, collect power-ups to boost the gameplay, and more.

The best features of Sonic Racing are as mentioned here:

Single-player, player vs. player (PvP) with friends, and PvP with global players

Create a team of mixed characters to defeat competitors

Five racing zones offering a total of 15 racing tracks

Artificial intelligence (AI) racer difficulty matched with your current racing tier

4. Skate City

If you’re into action sports, then you’ll love this skateboarding game app Skate City. If you think street skating is dangerous and want a virtual alternative, you can also try it out.

You’ll get complete access to the skate shop where you can customize your skateboard and skateboarding moves, skills, etc.

Moreover, if you like streaming your gameplay online, you can use its built-in streaming tools. You can grab still images of jaw-dropping skating moves and instantly post them from social media handles to engage with your fanbase.

5. Sonic Dash+

Sonic Dash+ is yet another one of the best Apple Arcade games from the makers of Sonic the Hedgehog.

It’s a mix of running and racing. Sonic the Hedgehog or other in-game characters run, dash, and jump through different landscapes full of hurdles and other challenges.

Some of the best in-game features are:

Loop de loops enable the in-game character to speed up in certain sections

Opportunity to unlock additional Sonic Team characters

6. Mini Motorways

You must aim to keep the traffic moving all the time. To aid you, there’ll be upgrades like roundabouts, highways, etc. If you like a stress-free real-time strategy game (RPG), you’ll love this game where you can apply your thought process to resolve evolving issues.

7. PAC-MAN Party Royale

PAC-MAN Party Royale is a retro-style arcade game with modern features like multiplayer, battle royale arena, PvP, and more. The battle royal event is a battle arena of four players. You can test your skills against global players through the multiplayer feature.

An exciting and challenging addition is the 256-glitch. The maze will collapse if no one becomes an evident winner soon enough. The safe area shrinks to a small compartment that can accommodate only one PAC-MAN, thus deciding the ultimate winner of the round.

8. Solitaire by MobilityWare+

However, the game isn’t compatible with your Mac or Apple TV devices. Its notable in-game functionalities are as outlined below:

Daily new challenges that earn you trophies and crowns

By leveling up, you can get a new title

Access exclusive stickers that you can share via iMessage

Get undo and hints for unlimited times; no need to buy in-app items

9. Asphalt 8: Airborne+

Another best Apple Arcade game is Asphalt 8: Airborne+. It’s a must-have game for you if you’re a fan of the Fast and Furious franchise of racing movies. Asphalt has been one of the leading names for high-speed racing games with premium visual effects.

With the help of the Apple GPU in devices like Mac, iPad, and iPhone, you can take the racing game experience to the next level. Some of the notable attractions of this game app are:

High-quality physics-based animation of vehicles, tracks, and racing environment

More than 2,300 decals to customize your racing car

Multiplayer gaming

Limited and premium events that pay more points and amazing rewards than regular racing events

10. The Oregon Trail

You’ll learn how to create a traveling party, fill the wagon with supplies, start traveling riding the wagon, face unknown challenges on the road, and more.

11. Crossy Road Castle

Crossy Road Castle is an endless spinning tower game. Here you must climb up through various towers and buildings to earn points. Cooperative or multiplayer climbing is the new addition to this retro arcade game app.

The multiplayer mode is available online and offline. For example, you can use a gaming console to play with your friends by connecting multiple gaming controllers offline.

Alternatively, you can set up an online multiplayer gameplay where each gamer will play from their iPhone, iPad, or Apple TV. Currently, Apple didn’t make this Apple Arcade game available for Mac.

Best Apple Arcade Games: Final Words

Now you know some of the best Apple Arcade games you must check out if you own an Apple Arcade subscription.

Next up, check out the best iPad games in 2023.

Best Robotic Process Automation Books You Must Read In 2023

Robotic Process Automation (RPA) has become an essential toolkit critical to enterprise success. An enterprise must first ensure which

Trilogy of RPA books

By Dr. Leslie P. Willcocks and Dr. Mary C. Lacity In this trilogy, the authors who are well versed in their writings, present an informative and well-balanced view of the subject. These books are a must-read by both the novice and automation practitioners. This series brings together-

Service Automation: Robots and the Future of Work

Robotic Process Automation and Risk Mitigation: The Definitive Guide

The RPA risk mitigation framework reveals key RPA risks and identifies 30 key risk mitigation practices which the research has found to be successful. This book serves as a definite key source of knowledge to all levels of the organization whether it is at the beginning of its RPA journey or has reached maturity levels.

Robotic Process and Cognitive Automation: The Next Phase

The book presents insights about the future of Automation with client interviews, providers, and analysts. The last book of the trilogy features a detailed analysis of the automation v/s the future of work debate. Readers can read an incisive and compelling, evidence-based perspective on the direction of service automation, and future trends through 2025. This book fuels the debate that automation technologies like Intelligent Automation and the newest Blockchain technologies are found to elevate human work rather than causing an elimination.  

The Robotic Process Automation Handbook: A Guide to Implementing RPA Systems

By Tom Taulli In this book, Tom Taulli explains how to leverage RPA effectively in enterprises to automate repetitive and rule-based processes, like data inputting and transferring, scheduling, cut and paste functions, filling out forms, and basic search function. Based on industry-specific case studies and industry best practices Tom Taulli explains how companies have been able to realize a substantial Return on Investment (ROI) with automation implementation, such as lessening the need for outsourcing or hiring.  

Robotic Process Automation: Guide to Building Software Robots, Automate Repetitive Tasks & Become an RPA Consultant

By Richard Murdoch This is the perfect book for those who are looking to become automation consultants. Automation career is a viable option for developers and non-developers due to the no-code developer environment and intuitive design interface.  

The Care and Feeding of Bots: An Owner’s Manual for Robotic Process Automation

By Christopher Surdak JD In this book, Christopher explains what it takes to make bots work in organizations. Author Chris Surdak summarizes his experiences gained by deploying hundreds of bots for dozens of organizations around the world over 5 years. Along the way, the number of failures, missteps, and errors are also shared with the readers. “The care and feeding of Bots” explains eighteen different ways in which bots can fail and how to avoid these failures. This book also discusses the next wave of cognitive automation which will be bought by artificial intelligence and Cognitive Computing.  

Robotic Process Automation RPA: A Complete Guide — 2023 Edition

By Gerardus Blokdyk This book lets an organization assess what is their backup plan taking labor costs and operational expenses into account. Robotic Process Automation RPA: A Complete Guide seeks answers to key questions like an organization’s outlook for top-line revenue growth over the next three years and the debate over bots taking up human work. Readers face 898 new and updated case-based questions, organized into seven core areas of automation implementation. This Self-Assessment will help enterprises identify areas in which Robotic Process Automation improvements can be made within their existing processes.  

Digital Workforce: Reduce Costs and Improve Efficiency Using Robotic Process Automation

By Rob King Rob king explains the different types of Robotic Process Automation and how to align enterprise needs to the solutions available to start an automation journey. This book gives key insights to scaling up as well. The book is a carefully considered approach that helps enterprises align their specific business needs to the need-based solution and the requisite business model.  

The Simple Implementation Guide to Robotic Process Automation: How to Best Implement RPA in an Organization

By Kelly Wibbenmeyer In this book, the author examines critical issues, like how to overcome common problems when implementing RPA and start an RPA implementation journey to further successfully carry it out; by obtaining funding and support from leaders to build an RPA team for future scaling up. The book includes the positives and negatives of various deployment strategies in addition to the key factors to be considered. The author explains in crisp timelines and uses cases giving a realistic view of how to manage business processes.  

Becoming Strategic with Robotic Process Automation

By Leslie P. Willcocks, John Hindle, and Mary C. Lacity

20 ‘Spelunky 2’ Tips You Must Know!

It’s hard to perfect perfection, but Mosssmouth managed to pull it off with Spelunky 2, the sequel to everybody’s favorite, deceptively maniacal rogue-like, Spelunky HD. The newest installment in the single most hectic mashup of Indiana Jones and just about everything else doubles down on the grueling fundamentals that made it the first one the mercilessly addictive game it was and does just enough tweaking to up the craziness without detracting from the vanilla experience.

It’s treasure, traps, and secrets galore in Spelunky 2, and to help you get the most out of each run we’ve compiled a list of tips and tricks in a complete Spelunky 2 guide down below.

What’s New in Spelunky 2?

Below we run through some of the bigger changes that came in with Spelunky 2. We’ll avoid going over minor changes and the more subtle QOL tweaks and stick to the changes that impact gameplay more significantly.

Fluid Dynamics

Lava, to water, to any sort of liquid in the game, Spelunky 2 now incorporates fluid dynamics in many components of the game to add an extra layer of complexity to the madness — so be sure to account for the behavior of liquids in the game. Because like everything else in Spelunky, they can probably kill you. Just make sure you know which way gravity pulls them, first.

Mounts

One of the coolest new additions with Spelunky 2 is the new Mounts. From turkeys to rock dogs to fire-breathing secret mounts, each one has their own movement mechanics and unique attacks — gotta catch ’em all!

Non-linear progression

Unlike Spelunky HD, the new Spelunky on the block actually incorporates branching levels. At key points in the game, players hit proverbial forks in the road which changes the immediate course of their run. Get to know these, because each level has its own unique constraints and caters to very different build-styles.

Items can Explode

Before you kick open the door of the shop and start blasting at the local shopkeeper, make sure to look at what he’s selling first. Items in Spelunky, worn, held, or just sitting around, can and will explode to great effect. This means being careful around flames while using jetpacks, and being wary around explosives while exercising your firearms.

The Whip’s been Nerfed

The old reliable of Spelunky weaponry, the Whip, has seen some significant changes this time around. It requires a little more finesse and getting used to, which we’ll go into with some depth later on, but can in fact be more useful if you know how to use it.

The Ghost is Buffed

Everybody’s favorite, highly impatient, insta-killing poltergeist is still around but this time he moves faster and can even split into multiple ghosts. So keep a move on.

New Secret Level

Like Spelunky HD’s super-secret Hell level, there exists an even more insane after-level reserved for only the mightiest gamers on Earth, known as the Cosmic Ocean. We won’t go into depth, since it’s a pretty massive spoiler, but if you want to learn how to get there you can read our special Spelunky 2 Cosmic Ocean Walkthrough.

Spelunky 2 Guide: 15 Tips and Tricks for Beginners and Advanced Players

1. Approach Shoplifting with Caution

While killing Shoplifters is an old mainstay of Spelunky, returning players and total newbies alike should approach the five-fingered discount with considerable caution this time around in Spelunky 2. Not only are shopkeepers tough as nails, they also have a pretty slim reaction time to any shoplifting and will come in hard. 

Also, note that many wares explode, so if you are going to start blasting (or provoke them to do so), be wary of any explosives laying around. That said, experiment, and once you get the hang of the subtleties of Spelunky combat, killing shopkeepers can be a profitable strategy indeed.

2. Make Use of the Sacrifice System!

The new Kali Altars reward you for sacrificing pets on their bloodstained pedestals with progressively better items — culminating in some excellent items like the Kapala that heals you with blood spilled by creatures.

3. Carry the Ghostpot to the Door

The Ghostpot contains a diamond worth 5K, but when you crack it open it spawns the unfriendliest ghost in the Spelunky universe that normally waits three minutes before ruining your day.

To circumvent this, carry the pot to the door to each level and crack it open right before heading out. Note that enemies and other hazards can just as easily crack it open along the way, so keep your eyes peeled. You know, more than usual.

4. There is Always a Shop on 1-2 or 1-3

Take note of the fact that there is always a shop on 1-2 or 1-3 and prepare accordingly. It’s completely randomized after that, so if you want to stock up before beginning your journey in earnest, it’s wise to either cash in a couple of diamonds or otherwise pad the wallet (or get ready to throw down with the shopkeeper for some “free” supplies).

5. Carry an Item at All Times

A good rule of thumb in Spelunky 2 is to always have an item. Anything and everything can come in handy. Even a rock. Especially a rock. You can throw items at enemies, use them to activate traps or break barricades, and generally find numerous ways to make them useful. You can’t carry bodies or large items between levels, but you can carry smaller things like rocks and arrows. This ties into our next tip:

6. Practice Throwing

Acquaint yourself with the “weight” and trajectory of thrown items in Spelunky 2. Not everything has the same throw distance or arc, so training your reflexes to account for these differences can be what keeps you alive on that glorious, personal record-breaking run.

7. Get the Udjat Eye

The Udjat Eye is nothing new to Spelunky. Like in the previous game, it grants players access to the Black Market (which is now just a sub-area, rather than its own level, and thus subject to the three-minute Ghost timer!) and reveals hidden treasure, but it is also used to unlock another secret area in one of the new biomes.

8. Use Tactical Head-Bounces

Ropes are great and all. They can be used to climb, descend safely, drop down into spikes or even be thrown at your enemies. And that’s why they should be used sparingly. A great way to reach a high ledge without using a rope when you don’t have a specialty item/mount is to hold up while bounce-killing an enemy. You can either try and time it or knock the enemy out and quickly place them where you’re going to make the jump.

9. Break Pots at a Distance

As containers, pots can be full of all sorts of wonderful items. Like gold, jewels, and venomous creatures. That last one might be more of an acquired taste. If you’re not a fan of scorpions, cobras, or even aliens, be sure to throw the pot a good distance away to break it open rather than smashing it open with your whip.

10. Don’t Free Fall

11. Turkeys Are Your Best Friend (and You Their Worst Enemy!)

Turkeys are a wonderful addition to Spelunky 2, and the first rideable mounts you’ll encounter in the game. Aside from being a huge boost to your movement dynamics, you can also be decidedly non-vegan and blow them up with a bomb or other source of fire to cook them. Cooked Turkeys increase your max HP by 1, making them worth the effort whenever an easy one comes your way.

12. Practice Cooking Bombs

Apart from learning how to throw, another key skill that veteran players of Spelunky swear by is the practice of “cooking” your bombs like a grenade. This involves dropping the bomb at your feet to start the 2.4-second timer, picking it up, and throwing it again — timing the explosion to detonate the bomb wherever you want within range.

It takes some serious practice and won’t be something you get the hang of immediately, but once you’re a well-oiled bomb-cooking machine you can basically turn any bomb into a poor man’s sticky bomb.

13. Back-whip

While whipping has been nerfed with a serious reduction in animation speed, many players might not be aware of the utility of the three-stage whip attack. The new whip attack actually strikes one tile behind and above you before striking in front of you — meaning if you really want to hit something quick, pull a trick-move and look the other way before swatting at your enemy.

14. Don’t Fear Spikes

Spikes will kill you a lot. A lot. But that doesn’t mean you have to be afraid of them. New Spelunky 2 players should note that you can actually descend safely into spikes with a rope, and actually walk into spikes from the side without a problem. The only way they kill is by falling on top of them.

15. You Can Whip Arrows (With Practice)

There are few things more annoying than Arrow Traps. Set on auto-fire, come anywhere within six tiles and you’re getting an arrow in the face. You can trigger the trap with an enemy, or a thrown item like a rock.

Even fancier, for those who don’t have anything to trigger the trap ahead of time, is to go full Geralt of Rivia and just deflect the thing with your whip. A well-timed whip will do the trick, but note that it takes practice. 

5 Spoiler-Ish Tips:

Below are a series of tips that involve some spoilers that newer players looking to organically explore the game may want to steer clear from. These involve specific discoveries that some people would rather stumble upon the old fashioned way: with a whole lot of death, despair, and eventual elation. 

1. Quillback’s Level Never Changes

2. Leprechauns Fulfill the Stereotype

A new mob, Leprechauns, can drop an item called 4-Leaf Clovers on death that sets the Ghost timer to five minutes and spawns a rainbow that can be bombed to reveal a pot of gold with a few thousand.

3. Vlad’s Got Enemies

On every 2-1 Volcana, there is a secret door that, when unlocked, reveals an imprisoned Van Horsing that will insta-kill Vlad the Vampire King for you later on if you rescue him. Just find the key, find Van Horsing, and let the geriatric vampire hunter do the rest.

4. Ghost Bling is Potent Bling

In 1-4, if you have Quillback destroy all of the bone piles (or bomb them all yourself), a secret door marked by a very faint bas relief of a Ghost’s face (same on that’s on the Ghostpot) will be revealed that can only be entered if you’re afflicted with a Curse.

As much as it sucks to be cursed, once inside you gain access to a “Ghost Shop” that contains 3 gift boxes for 20k each. The loot is completely randomized but can be as crazy as a Kapala alongside other useful loot, or something relatively mundane.

5. Most of Spelunky 2 is Almost Impossible to Reach

There is a secret after-level, called the Cosmic Ocean, that takes the powers of a demigod to reach, and the nigh-literal powers of 1.5 Gods to actually beat. If you’re interested in making the attempt, be sure to check out our complete, step-by-step walkthrough on how to reach that sacred and absolutely terrifying place.

Update the detailed information about Stemming Vs Lemmatization In Nlp: Must on the Cattuongwedding.com website. We hope the article's content will meet your needs, and we will regularly update the information to provide you with the fastest and most accurate information. Have a great day!