Page 300 - AI Ver 1.0 Class 10
P. 300

It is done using  WordNetLemmatizer package:

              >>>from nltk.stem.wordnet import WordNetLemmatizer

              >>>Lem = WordNetLemmatizer()

              >>>Lem.lemmatize("programming"))
              program


              Using NLTK Stopwords Corpus

              Stopwords such as is, am, are, this, a, an, the, etc. are insignificant in a sentence. It  is to be removed to avoid noise
              in a sentence.

              >>>from nltk.corpus import stopwords

              >>>Eng_stopwords = set(stopwords.words('english'))

              >>>print(Eng_stopwords)
              {'their', 'then', 'not', 'ma', 'here', 'other', 'won', 'up', 'weren', 'being', 'we', 'those', 'an', 'them', 'which', 'him', 'so',
              'yourselves', 'what', 'own', 'has', 'should', 'above', 'in', 'myself', 'against', 'that', 'before', 't', 'just', 'into', 'about', 'most',
              'd', 'where', 'our', 'or', 'such', 'ours', 'of', 'doesn', 'further', 'needn', 'now', 'some', 'too', 'hasn', 'more', 'the', 'yours',
              'her', 'below', 'same', 'how', 'very', 'is', 'did', 'you', 'his', 'when', 'few', 'does', 'down', 'yourself', 'i', 'do', 'both', 'shan',
              'have', 'itself', 'shouldn', 'through', 'themselves', 'o', 'didn', 've', 'm', 'off', 'out', 'but', 'and', 'doing', 'any', 'nor', 'over',
              'had', 'because', 'himself', 'theirs', 'me', 'by', 'she', 'whom', 'hers', 're', 'hadn', 'who', 'he', 'my', 'if', 'will', 'are', 'why',
              'from', 'am', 'with', 'been', 'its', 'ourselves', 'ain', 'couldn', 'a', 'aren', 'under', 'll', 'on', 'y', 'can', 'they', 'than', 'after',
              'wouldn', 'each', 'once', 'mightn', 'for', 'this', 'these', 's', 'only', 'haven', 'having', 'all', 'don', 'it', 'there', 'until', 'again',                                 Exercise
              'to', 'while', 'be', 'no', 'during', 'herself', 'as', 'mustn', 'between', 'was', 'at', 'your', 'were', 'isn', 'wasn'}

              For removing stopwords:
                                                                                                                                                                                     Solved Questions
              sentence_without_stopwords=[]
                                                                                                                                                                             SECTION A (Objective Type Questions)
              for t in sent_token:
                                                                                                                                                  uiz
                  if w not in Eng_stopwords:
                                                                                                                                            A.  Tick ( ) the correct option.
                      sentence_without_stopwords.append(w)                                                                                      1.  ………………………. is to be removed to avoid noise in a sentence.
              print(sentence_without_stopwords)                                                                                                   a.  Stemming                                       b.  Lemmatization
                                                                                                                                                  c.  Stopwords                                      d.  All of the above

                        At a Glance                                                                                                             2.  What is the role of NLP in AI Model?
                                                                                                                                                  a.  In NLP, modelling requires data pre-processing only after which the data is fed to the machine.
                    • Natural Language Processing or NLP is the subset of Artificial Intelligence that deals with how computers through
                    a program will perform tasks like speech recognition, translation, large amounts of Natural language data analysis            b.  Modeling in NLP helps in processing AI model.
                    and extraction so that a successful interaction can occur between the machines and the humans to give the desired             c.  Modelling is required to make an AI model.
                    output.                                                                                                                       d.  Modelling is used in simplification of data acquisition.
                    • The process of understanding human language is quite difficult for a machine. This process is divided into five
                    major steps.                                                                                                                3.  Where do we get the occurrence of each word and construct the vocabulary for the corpus?
                    • Automatic Text Summarization is the process of creating the most meaningful and relevant summary of voluminous              a.  Bag of words                                   b.  Stemming
                    texts from multiple resources.
                                                                                                                                                  c.  Dictionary                                     d.  lemmatization




                        298   Touchpad Artificial Intelligence-X
   295   296   297   298   299   300   301   302   303   304   305