Tokens in C++ | Types of Tokens in C++

Token is a smallest individual unit in a program. C++ program is written using these tokens, white spaces and syntax of C++ language. C++ tokens are similar to C tokens but some modifications is added in C++ tokens.

Types of Tokens in C++

Keywords

Identifiers

Constants

Variables

Operators

KEYWORDS

Keywords are reserved words and keywords cannot be used as variables or any user-defined functions names. Keywords are predefined words and their meaning is already known to the compiler. Many C++ language keywords are similar to C language keywords. ANSI C++ committee has added some more keywords in C++.

For example

int a;

Here int is a keyword. Meaning of int is already known to compiler.

IDENTIFIERS

The names taken by the programmer for creating variables, functions, arrays, classes etc are known as identifiers. C++ has its some own rules for naming identifiers. The rules for naming identifiers in C++ are

• Only alphabets, digits and underscores are allowed for identifiers.
• Digits are not allowed at the beginning of identifiers.
• Uppercase and lowercase letters are distinct
• Keywords cannot be used as variable’s name

All these rules are also same in C language.

CONSTANTS

Constants are fixed values that do not during the execution of a program. C++ support several types of literal constants.

Integer Constants

Integer constants are integers and they do not include decimal part. An integer constant can be either positive or negative.

Decimal integer constant (Base 10): Integer constants that do not start with 0. Example: 2, 67, -89 etc.

Octal integer constant (Base 8): Integer constants that starts with 0. Example: 089, 034 etc.

Hexadecimal integer constant (Base 16): Integer constants that starts with 0x or 0X. Example: 0XC

Floating Constants

Floating constants are also known as real constants. Floating constants contains decimal part. They can be either negative or positive. For example: 7.8, -2.1, 5.0, -1.0 etc.

Character Constants

Character constant is a single character enclosed by single quotes. A single character can be character constant. It can be letters, digits, operator and even space.

All these are character constant:

‘1’, ‘+’, ‘a’, ‘B’, ‘ ’

A character constant must contain only one character. For example, ‘-6’ is not character constant because it consists of two characters – and 6.

String constants

A string is a collection of characters. It is a sequence of character enclosed with double quotes.

For example: “PCTECHNICALPRO”

String is automatically appended by the special character called ‘\0’. ‘\0’ indicates end of string. So, the size of string is automatically increased by 1 bit because ‘\0’ is appended at the end of string.

PUNCTUATOR

Punctuators are most commonly used characters in C++.

For example: () {} [] , ; : * = #

() is used for function calls, arguments are placed in ().

{} indicates start and end of function body.

[] indicates single and multidimensional array subscript.

, is used to separate arguments in function declaration, definition and calling.

; is used to terminate statement.

: is used in conditional operator, scope resolution operator etc.

= is used for assigning values.

* is used in multiplication and pointer declaration.

# is used in pre-processor directives.

OPERATORS

Operator is used for manipulating an expression. 

C++ operators are:

Arithmetic Operators 

Increment / Decrement Operators

Relational Operators

Logical operators

Conditional Operators