pygments.scanner

This library implements a regex based scanner. Some languages like Pascal are easy to parse but have some keywords that depend on the context. Because of this it’s impossible to lex that just by using a regular expression lexer like the RegexLexer.

Have a look at the DelphiLexer to get an idea of how to use this scanner.

copyright:Copyright 2006-2013 by the Pygments team, see AUTHORS. license:BSD, see LICENSE for details.

exception pygments.scanner.EndOfText

Raise if end of text is reached and the user tried to call a match function.

class pygments.scanner.Scanner(text, flags=0)

Simple scanner

All method patterns are regular expression strings (not compiled expressions!)

check(pattern)

Apply pattern on the current position and return the match object. (Doesn’t touch pos). Use this for lookahead.

eos

True if the scanner reached the end of text.

get_char()

Scan exactly one char.

scan(pattern)

Scan the text for the given pattern and update pos/match and related fields. The return value is a boolen that indicates if the pattern matched. The matched value is stored on the instance as match, the last value is stored as last. start_pos is the position of the pointer before the pattern was matched, pos is the end position.

test(pattern)

Apply a pattern on the current position and check if it patches. Doesn’t touch pos.