#include <Lex.hpp>

Public Types
enum	Token { Keyword, Operator, Newline, Eof, String, Name, Integer, Double }

Public Member Functions
	Lex (Instance *inst)

Instance *	instance () const

void	DefineKeyword (const char *keyword)

void	DefineOperator (const char *op)

void	numbers_enabled (bool enabled)

void	Init (std::istream &in, bool single_line=false)

void	Close ()

void	DisableNewlineSkipping ()

void	EnableNewlineSkipping ()

bool	IsNext (Token token)

bool	IsNextKeyword (const char *keyword)

bool	IsNextOperator (const char *op)

Token	GetNextToken ()

std::string	GetNextTokenValue ()

void	Accept (Token token)

void	AcceptKeyword (const char *keyword)

void	AcceptOperator (const char *op)

void	AcceptFail ()
	No more options for accept are available. This accept always fails. More...

std::string	AcceptString ()

std::string	AcceptName ()

int	AcceptInteger ()

double	AcceptDouble ()

void	Recover (Token token)

void	RecoverKeyword (const char *keyword)

void	RecoverOperator (const char *op)

int	GetLine () const

int	GetColumn () const

void	LogicFail (const char *message)

void	ParserFail (const char *prefix=nullptr)

bool	Failed ()

Public Attributes
ClientMarker	cm

Detailed Description

Lexical analyzer and framework for simple LL parsers.

Definition at line 19 of file Lex.hpp.

Member Enumeration Documentation

◆ Token

enum iv::Lex::Token

Comments begin with # and end with newline.

Enumerator
Keyword
Operator
Newline
Eof
String
Name
Integer
Double

Definition at line 40 of file Lex.hpp.

Constructor & Destructor Documentation

◆ Lex()

iv::Lex::Lex ( Instance * inst )

Definition at line 13 of file Lex.cpp.

Member Function Documentation

◆ instance()

Instance * iv::Lex::instance ( ) const

Definition at line 30 of file Lex.cpp.

◆ DefineKeyword()

void iv::Lex::DefineKeyword ( const char * keyword )

Definition at line 35 of file Lex.cpp.

◆ DefineOperator()

void iv::Lex::DefineOperator ( const char * op )

Definition at line 40 of file Lex.cpp.

◆ numbers_enabled()

void iv::Lex::numbers_enabled ( bool enabled )

Numbers are enabled by default.

Definition at line 55 of file Lex.cpp.

◆ Init()

void iv::Lex::Init	(	std::istream &	in,
		bool	single_line = `false`
	)

Starts parsing of given input. First token is retrieved and pointer to stream is saved. Stream is released when Token::Eof is read or when Fail is called. So if we stop parsing before Eof is reached, Fail should be called. Param fileid is only used in error reports to tell user, which file contains given syntactic error.

Definition at line 60 of file Lex.cpp.

◆ Close()

void iv::Lex::Close ( )

Definition at line 77 of file Lex.cpp.

◆ DisableNewlineSkipping()

void iv::Lex::DisableNewlineSkipping ( )

Newline skipping is enabled by default. When newline skipping is enabled, the newlines will be processed (skipped) as whitespaces. Otherwise each newline is considered a specific token Newline and needs to be accepted before processing tokens that follow on next line.

Definition at line 45 of file Lex.cpp.

◆ EnableNewlineSkipping()

void iv::Lex::EnableNewlineSkipping ( )

Definition at line 50 of file Lex.cpp.

◆ IsNext()

bool iv::Lex::IsNext ( Token token )

Checks if next token is that what is given. Parser remembers which tokens were tested so they can be reported on potential Fail.

Definition at line 375 of file Lex.cpp.

◆ IsNextKeyword()

bool iv::Lex::IsNextKeyword ( const char * keyword )

Definition at line 393 of file Lex.cpp.

◆ IsNextOperator()

bool iv::Lex::IsNextOperator ( const char * op )

Definition at line 406 of file Lex.cpp.

◆ GetNextToken()

Lex::Token iv::Lex::GetNextToken ( )

Try not to use these two methods. They do not give information about acceptable values in errors. They also do not support the fact that Integers can be used in place where Double is requested.

Definition at line 419 of file Lex.cpp.

◆ GetNextTokenValue()

std::string iv::Lex::GetNextTokenValue ( )

Definition at line 427 of file Lex.cpp.

◆ Accept()

void iv::Lex::Accept ( Token token )

Accept - reads input and goes to next input. Might change FAIL flag to true.

Definition at line 443 of file Lex.cpp.

◆ AcceptKeyword()

void iv::Lex::AcceptKeyword ( const char * keyword )

Definition at line 478 of file Lex.cpp.

◆ AcceptOperator()

void iv::Lex::AcceptOperator ( const char * op )

Definition at line 508 of file Lex.cpp.

◆ AcceptFail()

void iv::Lex::AcceptFail ( )

No more options for accept are available. This accept always fails.

Definition at line 435 of file Lex.cpp.

◆ AcceptString()

std::string iv::Lex::AcceptString ( )

Definition at line 538 of file Lex.cpp.

◆ AcceptName()

std::string iv::Lex::AcceptName ( )

Definition at line 564 of file Lex.cpp.

◆ AcceptInteger()

int iv::Lex::AcceptInteger ( )

Definition at line 590 of file Lex.cpp.

◆ AcceptDouble()

double iv::Lex::AcceptDouble ( )

Definition at line 616 of file Lex.cpp.

◆ Recover()

void iv::Lex::Recover ( Token token )

This can be used to recover after a parser or logic fail (will not be effective in case of lexan or input fail). If Lex is not failed, this will do nothing. If Lex is failed, tokens will be read until specified token is found (or Eof occures). This token will be the next token (IsNext, can be Accepted) after recovery. In case of unrecoverable fail (IO or lexan fail), this will also have no effect.

Definition at line 758 of file Lex.cpp.

◆ RecoverKeyword()

void iv::Lex::RecoverKeyword ( const char * keyword )

Definition at line 785 of file Lex.cpp.

◆ RecoverOperator()

void iv::Lex::RecoverOperator ( const char * op )

Definition at line 799 of file Lex.cpp.

◆ GetLine()

int iv::Lex::GetLine ( ) const

Use this if there is possibility that this will be passed to LogicFail later.

Definition at line 649 of file Lex.cpp.

◆ GetColumn()

int iv::Lex::GetColumn ( ) const

Definition at line 654 of file Lex.cpp.

◆ LogicFail()

void iv::Lex::LogicFail ( const char * message )

Sets FAIL flag to true, reports current line and column and which is current token and reports which tokens were tested using IsNext* methods.

Definition at line 684 of file Lex.cpp.

◆ ParserFail()

void iv::Lex::ParserFail ( const char * prefix = nullptr )

Definition at line 693 of file Lex.cpp.

◆ Failed()

bool iv::Lex::Failed ( )

Returns value of a FAIL flag.

Definition at line 748 of file Lex.cpp.

Member Data Documentation

◆ cm

ClientMarker iv::Lex::cm

Definition at line 22 of file Lex.hpp.

The documentation for this class was generated from the following files:

ivorium_core/Parsing/Lex.hpp
ivorium_core/Parsing/Lex.cpp

Public Types

Public Member Functions

Public Attributes