Page 300 - AI Ver 1.0 Class 10
P. 300
It is done using WordNetLemmatizer package:
>>>from nltk.stem.wordnet import WordNetLemmatizer
>>>Lem = WordNetLemmatizer()
>>>Lem.lemmatize("programming"))
program
Using NLTK Stopwords Corpus
Stopwords such as is, am, are, this, a, an, the, etc. are insignificant in a sentence. It is to be removed to avoid noise
in a sentence.
>>>from nltk.corpus import stopwords
>>>Eng_stopwords = set(stopwords.words('english'))
>>>print(Eng_stopwords)
{'their', 'then', 'not', 'ma', 'here', 'other', 'won', 'up', 'weren', 'being', 'we', 'those', 'an', 'them', 'which', 'him', 'so',
'yourselves', 'what', 'own', 'has', 'should', 'above', 'in', 'myself', 'against', 'that', 'before', 't', 'just', 'into', 'about', 'most',
'd', 'where', 'our', 'or', 'such', 'ours', 'of', 'doesn', 'further', 'needn', 'now', 'some', 'too', 'hasn', 'more', 'the', 'yours',
'her', 'below', 'same', 'how', 'very', 'is', 'did', 'you', 'his', 'when', 'few', 'does', 'down', 'yourself', 'i', 'do', 'both', 'shan',
'have', 'itself', 'shouldn', 'through', 'themselves', 'o', 'didn', 've', 'm', 'off', 'out', 'but', 'and', 'doing', 'any', 'nor', 'over',
'had', 'because', 'himself', 'theirs', 'me', 'by', 'she', 'whom', 'hers', 're', 'hadn', 'who', 'he', 'my', 'if', 'will', 'are', 'why',
'from', 'am', 'with', 'been', 'its', 'ourselves', 'ain', 'couldn', 'a', 'aren', 'under', 'll', 'on', 'y', 'can', 'they', 'than', 'after',
'wouldn', 'each', 'once', 'mightn', 'for', 'this', 'these', 's', 'only', 'haven', 'having', 'all', 'don', 'it', 'there', 'until', 'again', Exercise
'to', 'while', 'be', 'no', 'during', 'herself', 'as', 'mustn', 'between', 'was', 'at', 'your', 'were', 'isn', 'wasn'}
For removing stopwords:
Solved Questions
sentence_without_stopwords=[]
SECTION A (Objective Type Questions)
for t in sent_token:
uiz
if w not in Eng_stopwords:
A. Tick ( ) the correct option.
sentence_without_stopwords.append(w) 1. ………………………. is to be removed to avoid noise in a sentence.
print(sentence_without_stopwords) a. Stemming b. Lemmatization
c. Stopwords d. All of the above
At a Glance 2. What is the role of NLP in AI Model?
a. In NLP, modelling requires data pre-processing only after which the data is fed to the machine.
• Natural Language Processing or NLP is the subset of Artificial Intelligence that deals with how computers through
a program will perform tasks like speech recognition, translation, large amounts of Natural language data analysis b. Modeling in NLP helps in processing AI model.
and extraction so that a successful interaction can occur between the machines and the humans to give the desired c. Modelling is required to make an AI model.
output. d. Modelling is used in simplification of data acquisition.
• The process of understanding human language is quite difficult for a machine. This process is divided into five
major steps. 3. Where do we get the occurrence of each word and construct the vocabulary for the corpus?
• Automatic Text Summarization is the process of creating the most meaningful and relevant summary of voluminous a. Bag of words b. Stemming
texts from multiple resources.
c. Dictionary d. lemmatization
298 Touchpad Artificial Intelligence-X

