allennlp.semparse.domain_languages

class allennlp.semparse.domain_languages.domain_language.BasicType(name: str)

Bases: allennlp.semparse.domain_languages.domain_language.PredicateType

A PredicateType representing a zero-argument predicate (which could technically be a function with no arguments or a constant; both are treated the same here).

class allennlp.semparse.domain_languages.domain_language.DomainLanguage(allowed_constants: Dict[str, Any] = None, start_types: Set[Type] = None)

Bases: object

A DomainLanguage specifies the functions available to use for a semantic parsing task. You write execution code for these functions, and we will automatically induce a grammar from those functions and give you a lisp interpreter that can use those functions. For example:

class Arithmetic(DomainLanguage):
    @predicate
    def add(self, num1: int, num2: int) -> int:
        return num1 + num2

    @predicate
    def halve(self, num: int) -> int:
        return num / 2

    ...

Instantiating this class now gives you a language object that can parse and execute logical forms, can convert logical forms to action sequences (linearized abstract syntax trees) and back again, and can list all valid production rules in a grammar induced from the specified functions.

>>> l = Arithmetic()
>>> l.execute("(add 2 3)")
5
>>> l.execute("(halve (add 12 4))")
8
>>> l.logical_form_to_action_sequence("(add 2 3)")
# See the docstring for this function for an description of what these strings mean.
['@start@ -> int', 'int -> [<int,int:int>, int, int]', '<int,int:int> -> add',
 'int -> 2', 'int -> 3']
>>> l.action_sequence_to_logical_form(l.logical_form_to_action_sequence('(add 2 3)'))
'(add 2 3)'
>>> l.get_nonterminal_productions()
{'<int,int:int>': ['add', 'divide', 'multiply', 'subtract'], '<int:int>': ['halve'], ...}

This is done with some reflection magic, with the help of the @predicate decorator and type annotations. For a method you define on a DomainLanguage subclass to be included in the language, it must be decorated with @predicate, and it must have type annotations on all arguments and on its return type. You can also add predicates and constants to the language using the add_predicate() and add_constant() functions, if you choose (minor point: constants with generic types (like Set[int]) must currently be specified as predicates, as the allowed_constants dictionary doesn’t pass along the generic type information).

The language we construct is purely functional - no defining variables or using lambda functions, or anything like that. If you would like to extend this code to handle more complex languages, open an issue on github.

We have rudimentary support for class hierarchies in the types that you provide. This is done through adding constants multiple times with different types. For example, say you have a Column class with NumberColumn and StringColumn subclasses. You can have functions that take the base class Column as an argument, and other functions that take the subclasses. These will get types like <List[Row],Column:List[str]> (for a “select” function that returns whatever cell text is in that column for the given rows), and <List[Row],NumberColumn,Number:List[Row]> (for a “greater_than” function that returns rows with a value in the column greater than the given number). These will generate argument types of Column and NumberColumn, respectively. NumberColumn is a subclass of Column, so we want the Column production to include all NumberColumns as options. This is done by calling add_constant() with each NumberColumn twice: once without a type_ argument (which infers the type as NumberColumn), and once with type_=Column. You can see a concrete example of how this works in the WikiTablesLanguage.

action_sequence_to_logical_form(self, action_sequence: List[str]) → str

Takes an action sequence as produced by logical_form_to_action_sequence(), which is a linearization of an abstract syntax tree, and reconstructs the logical form defined by that abstract syntax tree.

add_constant(self, name: str, value: Any, type_: Type = None)

Adds a constant to this domain language. You would typically just pass in a list of constants to the super().__init__() call in your constructor, but you can also call this method to add constants if it is more convenient.

Because we construct a grammar over this language for you, in order for the grammar to be finite we cannot allow arbitrary constants. Having a finite grammar is important when you’re doing semantic parsing - we need to be able to search over this space, and compute normalized probability distributions.

add_predicate(self, name: str, function: Callable, side_arguments: List[str] = None)

Adds a predicate to this domain language. Typically you do this with the @predicate decorator on the methods in your class. But, if you need to for whatever reason, you can also call this function yourself with a (type-annotated) function to add it to your language.

Parameters

namestr

The name that we will use in the induced language for this function.

functionCallable

The function that gets called when executing a predicate with the given name.

side_argumentsList[str], optional

If given, we will ignore these arguments for the purposes of grammar induction. This is to allow passing extra arguments from the decoder state that are not explicitly part of the language the decoder produces, such as the decoder’s attention over the question when a terminal was predicted. If you use this functionality, you also must use language.execute_action_sequence() instead of language.execute(), and you must pass the additional side arguments needed to that function. See execute_action_sequence() for more information.

all_possible_productions(self) → List[str]

Returns a sorted list of all production rules in the grammar induced by get_nonterminal_productions().

execute(self, logical_form: str)

Executes a logical form, using whatever predicates you have defined.

execute_action_sequence(self, action_sequence: List[str], side_arguments: List[Dict] = None)

Executes the program defined by an action sequence directly, without needing the overhead of translating to a logical form first. For any given program, execute() and this function are equivalent, they just take different representations of the program, so you can use whichever is more efficient.

Also, if you have state or side arguments associated with particular production rules (e.g., the decoder’s attention on an input utterance when a predicate was predicted), you must use this function to execute the logical form, instead of execute(), so that we can match the side arguments with the right functions.

get_nonterminal_productions(self) → Dict[str, List[str]]

Induces a grammar from the defined collection of predicates in this language and returns all productions in that grammar, keyed by the non-terminal they are expanding.

This includes terminal productions implied by each predicate as well as productions for the return type of each defined predicate. For example, defining a “multiply” predicate adds a “<int,int:int> -> multiply” terminal production to the grammar, and also a “int -> [<int,int:int>, int, int]” non-terminal production, because I can use the “multiply” predicate to produce an int.

is_nonterminal(self, symbol: str) → bool

Determines whether an input symbol is a valid non-terminal in the grammar.

logical_form_to_action_sequence(self, logical_form: str) → List[str]

Converts a logical form into a linearization of the production rules from its abstract syntax tree. The linearization is top-down, depth-first.

Each production rule is formatted as “LHS -> RHS”, where “LHS” is a single non-terminal type, and RHS is either a terminal or a list of non-terminals (other possible values for RHS in a more general context-free grammar are not produced by our grammar induction logic).

Non-terminals are types in the grammar, either basic types (like int, str, or some class that you define), or functional types, represented with angle brackets with a colon separating arguments from the return type. Multi-argument functions have commas separating their argument types. For example, <int:int> is a function that takes an integer and returns an integer, and <int,int:int> is a function that takes two integer arguments and returns an integer.

As an example translation from logical form to complete action sequence, the logical form (add 2 3) would be translated to ['@start@ -> int', 'int -> [<int,int:int>, int, int]', '<int,int:int> -> add', 'int -> 2', 'int -> 3'].

class allennlp.semparse.domain_languages.domain_language.FunctionType(argument_types: List[allennlp.semparse.domain_languages.domain_language.PredicateType], return_type: allennlp.semparse.domain_languages.domain_language.PredicateType)

Bases: allennlp.semparse.domain_languages.domain_language.PredicateType

A PredicateType representing a function with arguments. When seeing this as a string, it will be in angle brackets, with argument types separated by commas, and the return type separated from argument types with a colon. For example, def f(a: str) -> int: would look like <str:int>, and def g(a: int, b: int) -> int would look like <int,int:int>.

class allennlp.semparse.domain_languages.domain_language.PredicateType

Bases: object

A base class for types in a domain language. This serves much the same purpose as typing.Type, but we add a few conveniences to these types, so we construct separate classes for them and group them together under PredicateType to have a good type annotation for these types.

static get_function_type(arg_types: List[ForwardRef('PredicateType')], return_type: 'PredicateType') → 'PredicateType'

Constructs an NLTK ComplexType representing a function with the given argument and return types.

static get_type(type_: Type) → 'PredicateType'

Converts a python Type (as you might get from a type annotation) into a PredicateType. If the Type is callable, this will return a FunctionType; otherwise, it will return a BasicType.

BasicTypes have a single name parameter - we typically get this from type_.__name__. This doesn’t work for generic types (like List[str]), so we handle those specially, so that the name for the BasicType remains List[str], as you would expect.

allennlp.semparse.domain_languages.domain_language.get_generic_name(type_: Type) → str

allennlp.semparse.domain_languages.domain_language.is_callable(type_: Type) → bool

allennlp.semparse.domain_languages.domain_language.is_generic(type_: Type) → bool

allennlp.semparse.domain_languages.domain_language.nltk_tree_to_logical_form(tree: nltk.tree.Tree) → str

Given an nltk.Tree representing the syntax tree that generates a logical form, this method produces the actual (lisp-like) logical form, with all of the non-terminal symbols converted into the correct number of parentheses.

This is used in the logic that converts action sequences back into logical forms. It’s very unlikely that you will need this anywhere else.

allennlp.semparse.domain_languages.domain_language.predicate(function: Callable) → Callable

This is intended to be used as a decorator when you are implementing your DomainLanguage. This marks a function on a DomainLanguage subclass as a predicate that can be used in the language. See the DomainLanguage docstring for an example usage, and for what using this does.

allennlp.semparse.domain_languages.domain_language.predicate_with_side_args(side_arguments: List[str]) → Callable

Like predicate(), but used when some of the arguments to the function are meant to be provided by the decoder or other state, instead of from the language. For example, you might want to have a function use the decoder’s attention over some input text when a terminal was predicted. That attention won’t show up in the language productions. Use this decorator, and pass in the required state to DomainLanguage.execute_action_sequence(), if you need to ignore some arguments when doing grammar induction.

In order for this to work out, the side arguments must be after any non-side arguments. This is because we use *args to pass the non-side arguments, and **kwargs to pass the side arguments, and python requires that *args be before **kwargs.

This module defines a domain language for the QuaRel dataset, a simple domain theory for reasoning about qualitative relations.

class allennlp.semparse.domain_languages.quarel_language.Direction(number: int)

Bases: object

class allennlp.semparse.domain_languages.quarel_language.Property(name: str)

Bases: object

class allennlp.semparse.domain_languages.quarel_language.QuaRelLanguage

Bases: allennlp.semparse.domain_languages.domain_language.DomainLanguage

Domain language for the QuaRel dataset.

infer(self, setup: allennlp.semparse.domain_languages.quarel_language.QuaRelType, answer_0: allennlp.semparse.domain_languages.quarel_language.QuaRelType, answer_1: allennlp.semparse.domain_languages.quarel_language.QuaRelType) → int

Take the question and check if it is compatible with either of the answer choices.

class allennlp.semparse.domain_languages.quarel_language.QuaRelType(quarel_property: allennlp.semparse.domain_languages.quarel_language.Property, direction: allennlp.semparse.domain_languages.quarel_language.Direction, world: allennlp.semparse.domain_languages.quarel_language.World)

Bases: object

class allennlp.semparse.domain_languages.quarel_language.World(number: int)

Bases: object

allennlp.semparse.domain_languages.quarel_language.make_property_predicate(property_name: str) → Callable[[allennlp.semparse.domain_languages.quarel_language.Direction, allennlp.semparse.domain_languages.quarel_language.World], allennlp.semparse.domain_languages.quarel_language.QuaRelType]

class allennlp.semparse.domain_languages.nlvr_language.Box(objects_list: List[Dict[str, Any]], box_id: int)

Bases: object

This class represents each box containing objects in NLVR.

Parameters

objects_listList[JsonDict]

List of objects in the box, as given by the json file.

box_idint

An integer identifying the box index (0, 1 or 2).

class allennlp.semparse.domain_languages.nlvr_language.Color(color)

Bases: tuple

property color

Alias for field number 0

class allennlp.semparse.domain_languages.nlvr_language.NlvrLanguage(boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box])

Bases: allennlp.semparse.domain_languages.domain_language.DomainLanguage

above(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

Returns the set of objects in the same boxes that are above the given objects. That is, if the input is a set of two objects, one in each box, we will return a union of the objects above the first object in the first box, and those above the second object in the second box.

all_boxes(self) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

all_objects(self) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

below(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

Returns the set of objects in the same boxes that are below the given objects. That is, if the input is a set of two objects, one in each box, we will return a union of the objects below the first object in the first box, and those below the second object in the second box.

big(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

black(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

blue(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

bottom(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

Return the bottom most objects(i.e. maximum y_loc). The comparison is done separately for each box.

box_count_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → bool

box_count_greater(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → bool

box_count_greater_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → bool

box_count_lesser(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → bool

box_count_lesser_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → bool

box_count_not_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → bool

box_exists(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box]) → bool

circle(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

get_agenda_for_sentence(self, sentence: str) → List[str]

Given a sentence, returns a list of actions the sentence triggers as an agenda. The agenda can be used while by a parser to guide the decoder. sequences as possible. This is a simplistic mapping at this point, and can be expanded.

Parameters

sentencestr

The sentence for which an agenda will be produced.

medium(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

member_color_all_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], color: allennlp.semparse.domain_languages.nlvr_language.Color) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_color_any_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], color: allennlp.semparse.domain_languages.nlvr_language.Color) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_color_count_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_color_count_greater(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_color_count_greater_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_color_count_lesser(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_color_count_lesser_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_color_count_not_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_color_different(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box]) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_color_none_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], color: allennlp.semparse.domain_languages.nlvr_language.Color) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_color_same(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box]) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_count_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_count_greater(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_count_greater_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_count_lesser(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_count_lesser_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_count_not_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_shape_all_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], shape: allennlp.semparse.domain_languages.nlvr_language.Shape) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_shape_any_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], shape: allennlp.semparse.domain_languages.nlvr_language.Shape) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_shape_count_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_shape_count_greater(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_shape_count_greater_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_shape_count_lesser(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_shape_count_lesser_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_shape_count_not_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], count: int) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_shape_different(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box]) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_shape_none_equals(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box], shape: allennlp.semparse.domain_languages.nlvr_language.Shape) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

member_shape_same(self, boxes: Set[allennlp.semparse.domain_languages.nlvr_language.Box]) → Set[allennlp.semparse.domain_languages.nlvr_language.Box]

negate_filter(self, filter_function: Callable[[Set[allennlp.semparse.domain_languages.nlvr_language.Object]], Set[allennlp.semparse.domain_languages.nlvr_language.Object]]) → Callable[[Set[allennlp.semparse.domain_languages.nlvr_language.Object]], Set[allennlp.semparse.domain_languages.nlvr_language.Object]]

object_color_all_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], color: allennlp.semparse.domain_languages.nlvr_language.Color) → bool

object_color_any_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], color: allennlp.semparse.domain_languages.nlvr_language.Color) → bool

object_color_count_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_color_count_greater(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_color_count_greater_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_color_count_lesser(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_color_count_lesser_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_color_count_not_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_color_none_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], color: allennlp.semparse.domain_languages.nlvr_language.Color) → bool

object_count_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_count_greater(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_count_greater_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_count_lesser(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_count_lesser_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_count_not_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_exists(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → bool

object_in_box(self, box: Set[allennlp.semparse.domain_languages.nlvr_language.Box]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

object_shape_all_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], shape: allennlp.semparse.domain_languages.nlvr_language.Shape) → bool

object_shape_any_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], shape: allennlp.semparse.domain_languages.nlvr_language.Shape) → bool

object_shape_count_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_shape_count_greater(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_shape_count_greater_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_shape_count_lesser(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_shape_count_lesser_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_shape_count_not_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], count: int) → bool

object_shape_none_equals(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object], shape: allennlp.semparse.domain_languages.nlvr_language.Shape) → bool

same_color(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

Filters the set of objects, and returns those objects whose color is the most frequent color in the initial set of objects, if the highest frequency is greater than 1, or an empty set otherwise.

This is an unusual name for what the method does, but just as blue filters objects to those that are blue, this filters objects to those that are of the same color.

same_shape(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

Filters the set of objects, and returns those objects whose color is the most frequent color in the initial set of objects, if the highest frequency is greater than 1, or an empty set otherwise.

This is an unusual name for what the method does, but just as triangle filters objects to those that are triangles, this filters objects to those that are of the same shape.

small(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

square(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

top(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

Return the topmost objects (i.e. minimum y_loc). The comparison is done separately for each box.

touch_bottom(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

touch_corner(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

touch_left(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

touch_object(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

Returns all objects that touch the given set of objects.

touch_right(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

touch_top(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

touch_wall(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

triangle(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

yellow(self, objects: Set[allennlp.semparse.domain_languages.nlvr_language.Object]) → Set[allennlp.semparse.domain_languages.nlvr_language.Object]

class allennlp.semparse.domain_languages.nlvr_language.Object(attributes: Dict[str, Any], box_id: str)

Bases: object

Objects are the geometric shapes in the NLVR domain. They have values for attributes shape, color, x_loc, y_loc and size. We take a dict read from the JSON file and store it here, and define a get method for getting the attribute values. We need this to be hashable because need to make sets of Objects during execution, which get passed around between functions.

Parameters

attributesJsonDict

The dict for each object from the json file.

class allennlp.semparse.domain_languages.nlvr_language.Shape(shape)

Bases: tuple

property shape

Alias for field number 0

class allennlp.semparse.domain_languages.wikitables_language.Column(name)

Bases: tuple

property name

Alias for field number 0

class allennlp.semparse.domain_languages.wikitables_language.ComparableColumn(name)

Bases: allennlp.semparse.domain_languages.wikitables_language.Column

class allennlp.semparse.domain_languages.wikitables_language.DateColumn(name)

Bases: allennlp.semparse.domain_languages.wikitables_language.ComparableColumn

class allennlp.semparse.domain_languages.wikitables_language.NumberColumn(name)

Bases: allennlp.semparse.domain_languages.wikitables_language.ComparableColumn

class allennlp.semparse.domain_languages.wikitables_language.Row(values)

Bases: tuple

property values

Alias for field number 0

class allennlp.semparse.domain_languages.wikitables_language.StringColumn(name)

Bases: allennlp.semparse.domain_languages.wikitables_language.Column

class allennlp.semparse.domain_languages.wikitables_language.WikiTablesLanguage(table_context: allennlp.semparse.contexts.table_question_context.TableQuestionContext)

Bases: allennlp.semparse.domain_languages.domain_language.DomainLanguage

Implements the functions in the variable free language we use, that’s inspired by the one in “Memory Augmented Policy Optimization for Program Synthesis with Generalization” by Liang et al.

Because some of the functions are only allowed if some conditions hold on the table, we don’t use the @predicate decorator for all of the language functions. Instead, we add them to the language using add_predicate if, e.g., there is a column with dates in it.

all_rows(self) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

argmax(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.ComparableColumn) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

Takes a list of rows and a column name and returns a list containing a single row (dict from columns to cells) that has the maximum numerical value in the given column. We return a list instead of a single dict to be consistent with the return type of select and all_rows.

argmin(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.ComparableColumn) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

Takes a list of rows and a column and returns a list containing a single row (dict from columns to cells) that has the minimum numerical value in the given column. We return a list instead of a single dict to be consistent with the return type of select and all_rows.

average(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.NumberColumn) → numbers.Number

Takes a list of rows and a column and returns the mean of the values under that column in those rows.

count(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row]) → numbers.Number

date(self, year: numbers.Number, month: numbers.Number, day: numbers.Number) → allennlp.semparse.common.date.Date

Takes three numbers and returns a Date object whose year, month, and day are the three numbers in that order.

diff(self, first_row: List[allennlp.semparse.domain_languages.wikitables_language.Row], second_row: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.NumberColumn) → numbers.Number

Takes a two rows and a number column and returns the difference between the values under that column in those two rows.

evaluate_action_sequence(self, action_sequence: List[str], target_list: List[str]) → bool

Similar to evaluate_logical_form except that it takes an action sequence instead. The reason this is separate is because there is a separate method DomainLanguage.execute_action_sequence that executes the action sequence directly.

evaluate_denotation(self, denotation: Any, target_list: List[str]) → bool

Compares denotation with a target list and returns whether they are both the same according to the official evaluator.

evaluate_logical_form(self, logical_form: str, target_list: List[str]) → bool

Takes a logical form, and the list of target values as strings from the original lisp string, and returns True iff the logical form executes to the target list, using the official WikiTableQuestions evaluation script.

filter_date_equals(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.DateColumn, filter_value: allennlp.semparse.common.date.Date) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

filter_date_greater(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.DateColumn, filter_value: allennlp.semparse.common.date.Date) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

filter_date_greater_equals(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.DateColumn, filter_value: allennlp.semparse.common.date.Date) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

filter_date_lesser(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.DateColumn, filter_value: allennlp.semparse.common.date.Date) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

filter_date_lesser_equals(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.DateColumn, filter_value: allennlp.semparse.common.date.Date) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

filter_date_not_equals(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.DateColumn, filter_value: allennlp.semparse.common.date.Date) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

filter_in(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.StringColumn, filter_values: List[str]) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

filter_not_in(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.StringColumn, filter_values: List[str]) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

filter_number_equals(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.NumberColumn, filter_value: numbers.Number) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

filter_number_greater(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.NumberColumn, filter_value: numbers.Number) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

filter_number_greater_equals(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.NumberColumn, filter_value: numbers.Number) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

filter_number_lesser(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.NumberColumn, filter_value: numbers.Number) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

filter_number_lesser_equals(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.NumberColumn, filter_value: numbers.Number) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

filter_number_not_equals(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.NumberColumn, filter_value: numbers.Number) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

first(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row]) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

Takes an expression that evaluates to a list of rows, and returns the first one in that list.

get_agenda(self, conservative: bool = False)

Returns an agenda that can be used guide search.

Parameters

conservativebool

Setting this flag will return a subset of the agenda items that correspond to high confidence lexical matches. You’ll need this if you are going to use this agenda to penalize a model for producing logical forms that do not contain some items in it. In that case, you’ll want this agenda to have close to perfect precision, at the cost of a lower recall. You may not want to set this flag if you are sorting the output from a search procedure based on how much of this agenda is satisfied.

static is_instance_specific_entity(entity_name: str) → bool

Instance specific entities are column names, strings and numbers. Returns True if the entity is one of those.

last(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row]) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

Takes an expression that evaluates to a list of rows, and returns the last one in that list.

max_date(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.DateColumn) → allennlp.semparse.common.date.Date

Takes a list of rows and a column and returns the max of the values under that column in those rows.

max_number(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.NumberColumn) → numbers.Number

Takes a list of rows and a column and returns the max of the values under that column in those rows.

min_date(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.DateColumn) → allennlp.semparse.common.date.Date

Takes a list of rows and a column and returns the min of the values under that column in those rows.

min_number(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.NumberColumn) → numbers.Number

Takes a list of rows and a column and returns the min of the values under that column in those rows.

mode_date(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.DateColumn) → allennlp.semparse.common.date.Date

Takes a list of rows and a column and returns the most frequent value under that column in those rows.

mode_number(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.NumberColumn) → numbers.Number

Takes a list of rows and a column and returns the most frequent value under that column in those rows.

mode_string(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.StringColumn) → List[str]

Takes a list of rows and a column and returns the most frequent values (one or more) under that column in those rows.

next(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row]) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

Takes an expression that evaluates to a single row, and returns the row that occurs after the input row in the original set of rows. If the input row happens to be the last row, we will return an empty list.

previous(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row]) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

Takes an expression that evaluates to a single row, and returns the row that occurs before the input row in the original set of rows. If the input row happens to be the top row, we will return an empty list.

same_as(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.Column) → List[allennlp.semparse.domain_languages.wikitables_language.Row]

Takes a row and a column and returns a list of rows from the full set of rows that contain the same value under the given column as the given row.

select_date(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.DateColumn) → allennlp.semparse.common.date.Date

Select function takes a row as a list and a column name and returns the date in that column.

select_number(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.NumberColumn) → numbers.Number

Select function takes a row (as a list) and a column name and returns the number in that column. If multiple rows are given, will return the first number that is not None.

select_string(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.StringColumn) → List[str]

Select function takes a list of rows and a column name and returns a list of strings as in cells.

sum(self, rows: List[allennlp.semparse.domain_languages.wikitables_language.Row], column: allennlp.semparse.domain_languages.wikitables_language.NumberColumn) → numbers.Number

Takes a list of rows and a column and returns the sum of the values under that column in those rows.