Avro C#

Classes | Public Types | Public Member Functions | Static Public Member Functions | Protected Member Functions | Static Protected Member Functions | Properties | List of all members
Avro.IO.Parsing.Symbol Class Reference

Symbol is the base of all symbols (terminals and non-terminals) of the grammar. More...

Inheritance diagram for Avro.IO.Parsing.Symbol:

## Classes
class | Alternative
| Alternative symbol. More...
class | DefaultStartAction
| The default start action. More...
class | EnumLabelsAction
| The enum labels action. More...
class | ErrorAction
| The error action. More...
class | FieldAdjustAction
| The field adjust action. More...
class | FieldOrderAction
| THe field order action. More...
class | Fixup
| Fixup symbol. More...
class | ImplicitAction
| Implicit action. More...
class | IntCheckAction
| Int check action. More...
class | Repeater
| Repeater symbol. More...
class | ResolvingAction
| The resolving action. More...
class | Root
| Root symbol. More...
class | Sequence
| Sequence symbol. More...
class | SkipAction
| The skip action. More...
class | Terminal
| Terminal symbol. More...
class | UnionAdjustAction
| The union adjust action. More...
class | WriterUnionAction
| The writer union action. More...

## Public Types
enum | Kind { Kind.Terminal, Kind.Root, Kind.Sequence, Kind.Repeater, Kind.Alternative, Kind.ImplicitAction, Kind.ExplicitAction }
| The type of symbol. More...

## Public Member Functions
virtual int | FlattenedSize ()
| Returns the flattened size. More...

## Static Public Member Functions
static Symbol | NewRoot (params Symbol[] symbols)
| A convenience method to construct a root symbol. More...
static Symbol | NewSeq (params Symbol[] production)
| A convenience method to construct a sequence. More...
static Symbol | NewRepeat (Symbol endSymbol, params Symbol[] symsToRepeat)
| A convenience method to construct a repeater. More...
static Symbol | NewAlt (Symbol[] symbols, string[] labels)
| A convenience method to construct a union. More...

## Protected Member Functions
| Symbol (Kind kind)
| Constructs a new symbol of the given kind. More...
| Symbol (Kind kind, Symbol[] production)
| Constructs a new symbol of the given kind and production. More...
virtual Symbol | Flatten (IDictionary< Sequence, Sequence > map, IDictionary< Sequence, IList< Fixup >> map2)
| Flatten the given sub-array of symbols into a sub-array of symbols. More...

## Static Protected Member Functions
static Symbol | Error (string e)
| A convenience method to construct an ErrorAction. More...
static Symbol | Resolve (Symbol w, Symbol r)
| A convenience method to construct a ResolvingAction. More...
static void | Flatten (Symbol[] input, int start, Symbol[] output, int skip, IDictionary< Sequence, Sequence > map, IDictionary< Sequence, IList< Fixup >> map2)
| Flattens the given sub-array of symbols into an sub-array of symbols. Every `Sequence` in the input are replaced by its production recursively. Non-`Sequence` symbols, they internally have other symbols those internal symbols also get flattened. When flattening is done, the only place there might be Sequence symbols is in the productions of a Repeater, Alternative, or the symToParse and symToSkip in a UnionAdjustAction or SkipAction. More...
static int | FlattenedSize (Symbol[] symbols, int start)
| Returns the amount of space required to flatten the given sub-array of symbols. More...

## Properties
Kind | SymKind`[get]`
| The kind of this symbol.
Symbol[] | Production`[get]`
| The production for this symbol. If this symbol is a terminal this is `null`. Otherwise this holds the the sequence of the symbols that forms the production for this symbol. The sequence is in the reverse order of production. This is useful for easy copying onto parsing stack. More...
static Symbol | Null = new Terminal("null")`[get]`
| The terminal symbols for the grammar. More...
static Symbol | Boolean = new Terminal("boolean")`[get]`
| Boolean More...
static Symbol | Int = new Terminal("int")`[get]`
| Int More...
static Symbol | Long = new Terminal("long")`[get]`
| Long More...
static Symbol | Float = new Terminal("float")`[get]`
| Float More...
static Symbol | Double = new Terminal("double")`[get]`
| Double More...
static Symbol | String = new Terminal("string")`[get]`
| String More...
static Symbol | Bytes = new Terminal("bytes")`[get]`
| Bytes More...
static Symbol | Fixed = new Terminal("fixed")`[get]`
| Fixed More...
static Symbol | Enum = new Terminal("enum")`[get]`
| Enum More...
static Symbol | Union = new Terminal("union")`[get]`
| Union More...
static Symbol | ArrayStart = new Terminal("array-start")`[get]`
| ArrayStart More...
static Symbol | ArrayEnd = new Terminal("array-end")`[get]`
| ArrayEnd More...
static Symbol | MapStart = new Terminal("map-start")`[get]`
| MapStart More...
static Symbol | MapEnd = new Terminal("map-end")`[get]`
| MapEnd More...
static Symbol | ItemEnd = new Terminal("item-end")`[get]`
| ItemEnd More...
static Symbol | WriterUnion = new WriterUnionAction()`[get]`
| WriterUnion More...
static Symbol | FieldAction = new Terminal("field-action")`[get]`
| FieldAction - a pseudo terminal used by parsers More...
static Symbol | RecordStart = new ImplicitAction(false)`[get]`
| RecordStart More...
static Symbol | RecordEnd = new ImplicitAction(true)`[get]`
| RecordEnd More...
static Symbol | UnionEnd = new ImplicitAction(true)`[get]`
| UnionEnd More...
static Symbol | FieldEnd = new ImplicitAction(true)`[get]`
| FieldEnd More...
static Symbol | DefaultEndAction = new ImplicitAction(true)`[get]`
| DefaultEndAction More...
static Symbol | MapKeyMarker = new Terminal("map-key-marker")`[get]`
| MapKeyMarker More...

## Detailed Description

Symbol is the base of all symbols (terminals and non-terminals) of the grammar.

## Member Enumeration Documentation

## ◆ Kind

enum Avro.IO.Parsing.Symbol.Kind
| strong

The type of symbol.

Enumerator
Terminal | terminal symbols which have no productions
Root | Start symbol for some grammar
Sequence | non-terminal symbol which is a sequence of one or more other symbols
Repeater | non-terminal to represent the contents of an array or map
Alternative | non-terminal to represent the union
ImplicitAction | non-terminal action symbol which are automatically consumed
ExplicitAction | non-terminal action symbol which is explicitly consumed

## Constructor & Destructor Documentation

## ◆ Symbol() [1/2]

Avro.IO.Parsing.Symbol.Symbol | ( | Kind | kind | ) |
| inline protected

Constructs a new symbol of the given kind.

## ◆ Symbol() [2/2]

Avro.IO.Parsing.Symbol.Symbol | ( | Kind | kind,
|  | Symbol[] | production
| ) |  |
| inline protected

Constructs a new symbol of the given kind and production.

## Member Function Documentation

## ◆ Error()

static Symbol Avro.IO.Parsing.Symbol.Error | ( | string | e | ) |
| static protected

A convenience method to construct an ErrorAction.

Parameters

  e |

## ◆ Flatten() [1/2]

virtual Symbol Avro.IO.Parsing.Symbol.Flatten | ( | IDictionary< Sequence, Sequence > | map,
|  | IDictionary< Sequence, IList< Fixup >> | map2
| ) |  |
| protected virtual

Flatten the given sub-array of symbols into a sub-array of symbols.

Reimplemented in Avro.IO.Parsing.Symbol.UnionAdjustAction, Avro.IO.Parsing.Symbol.SkipAction, Avro.IO.Parsing.Symbol.ResolvingAction, Avro.IO.Parsing.Symbol.Alternative, Avro.IO.Parsing.Symbol.Repeater, and Avro.IO.Parsing.Symbol.Sequence.

## ◆ Flatten() [2/2]

static void Avro.IO.Parsing.Symbol.Flatten | ( | Symbol[] | input,
|  | int | start,
|  | Symbol[] | output,
|  | int | skip,
|  | IDictionary< Sequence, Sequence > | map,
|  | IDictionary< Sequence, IList< Fixup >> | map2
| ) |  |
| inline static protected

Flattens the given sub-array of symbols into an sub-array of symbols. Every `Sequence` in the input are replaced by its production recursively. Non-`Sequence` symbols, they internally have other symbols those internal symbols also get flattened. When flattening is done, the only place there might be Sequence symbols is in the productions of a Repeater, Alternative, or the symToParse and symToSkip in a UnionAdjustAction or SkipAction.

Why is this done? We want our parsers to be fast. If we left the grammars unflattened, then the parser would be constantly copying the contents of nested Sequence productions onto the parsing stack. Instead, because of flattening, we have a long top-level production with no Sequences unless the Sequence is absolutely needed, e.g., in the case of a Repeater or an Alternative.

Well, this is not exactly true when recursion is involved. Where there is a recursive record, that record will be "inlined" once, but any internal (ie, recursive) references to that record will be a Sequence for the record. That Sequence will not further inline itself – it will refer to itself as a Sequence. The same is true for any records nested in this outer recursive record. Recursion is rare, and we want things to be fast in the typical case, which is why we do the flattening optimization.

The algorithm does a few tricks to handle recursive symbol definitions. In order to avoid infinite recursion with recursive symbols, we have a map of Symbol->Symbol. Before fully constructing a flattened symbol for a `Sequence` we insert an empty output symbol into the map and then start filling the production for the `Sequence`. If the same `Sequence` is encountered due to recursion, we simply return the (empty) output `Sequence` from the map. Then we actually fill out the production for the `Sequence`. As part of the flattening process we copy the production of `Sequence`s into larger arrays. If the original `Sequence` has not not be fully constructed yet, we copy a bunch of `null`s. Fix-up remembers all those `null` patches. The fix-ups gets finally filled when we know the symbols to occupy those patches.

Parameters

  input | The array of input symbols to flatten
  start | The position where the input sub-array starts.
  output | The output that receives the flattened list of symbols. The output array should have sufficient space to receive the expanded sub-array of symbols.
  skip | The position where the output input sub-array starts.
  map | A map of symbols which have already been expanded. Useful for handling recursive definitions and for caching.
  map2 | A map to to store the list of fix-ups.

## ◆ FlattenedSize() [1/2]

virtual int Avro.IO.Parsing.Symbol.FlattenedSize | ( |  | ) |
| virtual

Returns the flattened size.

Reimplemented in Avro.IO.Parsing.Symbol.Sequence.

## ◆ FlattenedSize() [2/2]

static int Avro.IO.Parsing.Symbol.FlattenedSize | ( | Symbol[] | symbols,
|  | int | start
| ) |  |
| inline static protected

Returns the amount of space required to flatten the given sub-array of symbols.

Parameters

  symbols | The array of input symbols.
  start | The index where the subarray starts.

Returns
  The number of symbols that will be produced if one expands the given input.

## ◆ NewAlt()

static Symbol Avro.IO.Parsing.Symbol.NewAlt | ( | Symbol[] | symbols,
|  | string[] | labels
| ) |  |
| static

A convenience method to construct a union.

## ◆ NewRepeat()

static Symbol Avro.IO.Parsing.Symbol.NewRepeat | ( | Symbol | endSymbol,
|  | params Symbol[] | symsToRepeat
| ) |  |
| static

A convenience method to construct a repeater.

Parameters

  endSymbol | The end symbol.
  symsToRepeat | The symbols to repeat in the repeater.

## ◆ NewRoot()

static Symbol Avro.IO.Parsing.Symbol.NewRoot | ( | params Symbol[] | symbols | ) |
| static

A convenience method to construct a root symbol.

## ◆ NewSeq()

static Symbol Avro.IO.Parsing.Symbol.NewSeq | ( | params Symbol[] | production | ) |
| static

A convenience method to construct a sequence.

Parameters

  production | The constituent symbols of the sequence.

## ◆ Resolve()

static Symbol Avro.IO.Parsing.Symbol.Resolve | ( | Symbol | w,
|  | Symbol | r
| ) |  |
| static protected

A convenience method to construct a ResolvingAction.

Parameters

  w | The writer symbol
  r | The reader symbol

## Property Documentation

## ◆ ArrayEnd

Symbol Avro.IO.Parsing.Symbol.ArrayEnd = new Terminal("array-end")
| static get

ArrayEnd

## ◆ ArrayStart

Symbol Avro.IO.Parsing.Symbol.ArrayStart = new Terminal("array-start")
| static get

ArrayStart

## ◆ Boolean

Symbol Avro.IO.Parsing.Symbol.Boolean = new Terminal("boolean")
| static get

Boolean

## ◆ Bytes

Symbol Avro.IO.Parsing.Symbol.Bytes = new Terminal("bytes")
| static get

Bytes

## ◆ DefaultEndAction

Symbol Avro.IO.Parsing.Symbol.DefaultEndAction = new ImplicitAction(true)
| static get

DefaultEndAction

## ◆ Double

Symbol Avro.IO.Parsing.Symbol.Double = new Terminal("double")
| static get

Double

## ◆ Enum

Symbol Avro.IO.Parsing.Symbol.Enum = new Terminal("enum")
| static get

Enum

## ◆ FieldAction

Symbol Avro.IO.Parsing.Symbol.FieldAction = new Terminal("field-action")
| static get

FieldAction - a pseudo terminal used by parsers

## ◆ FieldEnd

Symbol Avro.IO.Parsing.Symbol.FieldEnd = new ImplicitAction(true)
| static get

FieldEnd

## ◆ Fixed

Symbol Avro.IO.Parsing.Symbol.Fixed = new Terminal("fixed")
| static get

Fixed

## ◆ Float

Symbol Avro.IO.Parsing.Symbol.Float = new Terminal("float")
| static get

Float

## ◆ Int

Symbol Avro.IO.Parsing.Symbol.Int = new Terminal("int")
| static get

Int

## ◆ ItemEnd

Symbol Avro.IO.Parsing.Symbol.ItemEnd = new Terminal("item-end")
| static get

ItemEnd

## ◆ Long

Symbol Avro.IO.Parsing.Symbol.Long = new Terminal("long")
| static get

Long

## ◆ MapEnd

Symbol Avro.IO.Parsing.Symbol.MapEnd = new Terminal("map-end")
| static get

MapEnd

## ◆ MapKeyMarker

Symbol Avro.IO.Parsing.Symbol.MapKeyMarker = new Terminal("map-key-marker")
| static get

MapKeyMarker

## ◆ MapStart

Symbol Avro.IO.Parsing.Symbol.MapStart = new Terminal("map-start")
| static get

MapStart

## ◆ Null

Symbol Avro.IO.Parsing.Symbol.Null = new Terminal("null")
| static get

The terminal symbols for the grammar.

## ◆ Production

Symbol [] Avro.IO.Parsing.Symbol.Production
| get

The production for this symbol. If this symbol is a terminal this is `null`. Otherwise this holds the the sequence of the symbols that forms the production for this symbol. The sequence is in the reverse order of production. This is useful for easy copying onto parsing stack.

Please note that this is a final. So the production for a symbol should be known before that symbol is constructed. This requirement cannot be met for those symbols which are recursive (e.g. a record that holds union a branch of which is the record itself). To resolve this problem, we initialize the symbol with an array of nulls. Later we fill the symbols. Not clean, but works. The other option is to not have this field a final. But keeping it final and thus keeping symbol immutable gives some comfort. See various generators how we generate records.

## ◆ RecordEnd

Symbol Avro.IO.Parsing.Symbol.RecordEnd = new ImplicitAction(true)
| static get

RecordEnd

## ◆ RecordStart

Symbol Avro.IO.Parsing.Symbol.RecordStart = new ImplicitAction(false)
| static get

RecordStart

## ◆ String

Symbol Avro.IO.Parsing.Symbol.String = new Terminal("string")
| static get

String

## ◆ Union

Symbol Avro.IO.Parsing.Symbol.Union = new Terminal("union")
| static get

Union

## ◆ UnionEnd

Symbol Avro.IO.Parsing.Symbol.UnionEnd = new ImplicitAction(true)
| static get

UnionEnd

## ◆ WriterUnion

Symbol Avro.IO.Parsing.Symbol.WriterUnion = new WriterUnionAction()
| static get

WriterUnion

The documentation for this class was generated from the following file:

- src/apache/main/IO/Parsing/Symbol.cs