1. What are tokens in Python ? How many types of tokens are allowed in Python ? Examplify your answer.

The smallest individual unit in a program is known as a token. There are five types of tokens allowed in Python. They are :

Keywords : for, del, elif, else etc.

Identifiers : Variable names like balance, class names like Vehicle etc

Literals : String, Numeric, Boolean like ‘abcd’, None etc

Operators : Unary, Binary, Bitwise like ‘+’, ‘&’, ‘^’ etc

Punctuators : Symbols like ‘#’, ‘(‘, ‘[‘, ‘=’ etc.