Page 142 - Computer Science V2.0 Class 11

6.2 Tokens

The word lexical refers to the vocabulary of a language. A statement in Python is a sequence of its lexical components,
called tokens, such as numbers, parentheses, and commas. Python has the following tokens (to be discussed in this
section): keywords, identifiers, literals, delimiters, and operators (see Fig 6.1).

Keywords

Operators Identifiers
Python
Tokens

Delimeters Literals

Fig 6.1: Tokens in Python

6.2.1 Keywords
Keywords (also called reserved words) are words whose meaning and usage is predefined in a programming language.
Thus, a keyword cannot be used for any purpose other than the one defined by the language. Every programming
language has a set of keywords. Some of the most commonly used keywords are shown in Table 6.2. note that the
keywords are case-sensitive. For example, while True is a keyword true is not.
Table 6.2: Keywords in Python

False None True and
as assert async await
break class continue def
del elif else except
finally for from global
if import in is
lambda nonlocal not or
pass raise return try
while with yield
6.2.2 Identifiers

An identifier is a name for a Python object. For example, the identifiers name, age, rollNo, and marks may be
used to refer to the name, age, roll number, and marks of a student. Like any other programming language, Python
follows certain rules for framing identifiers, which are as follows:
● An identifier should begin with an uppercase or lowercase alphabet, or an underscore (_).
● The first letter of the identifier (including an underscore) may be followed by an arbitrary combination of alphabets
(a-z, A-Z), digits (0-9), and underscore (_).
Below we make some observations and give some suggestions for constructing identifiers:
● It is clear from the above rules that an identifier cannot start with a digit.
● Although an identifier may be of any length, we prefer to keep it short and meaningful. For example, to denote roll

number, name, and marks of a student, we may use rollNo, name, and marks as identifiers rather than a, b,
and c, which do not relate them to the purpose for which they are used.

128 Touchpad Computer Science (Ver. 2.0)-XI