Regular Expressions for Language Model Completions.

Some people, when confronted with a problem, think “I know, I'll use regular expressions.” Now they have two problems.

Exact structure out of any language model completion with regular expressions.

Return specific syntactic structure (e.g. JSON or XML), or specific semantic structure (e.g. a date or a number), or even complete templates (e.g. a sentence with a blank to fill in).

How does it work? ReLLM filters non-matching tokens pre-generation. For each token, ReLLM tests every possible completion against a partial regex. For the potential completions that do not match the pattern, ReLLM masks the logits so that the language model does not generate them.

If you are looking for a hosted version of ReLLM, check out the Thiggle Regex Completion API at github.com/thiggle/api

pip install rellm

The preliminary results are interesting -- even for small models, constraining the token space with ReLLM can improve the quality of the completions. Not to mention the ability to more easily parse the output programmatically. Take a look at some of the examples. For an example of parsing a context-free grammar (like JSON) with ReLLM, see r2d4/parserllm.

import regex
from transformers import AutoModelForCausalLM, AutoTokenizer

from rellm import complete_re

model = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer = AutoTokenizer.from_pretrained("gpt2")

prompt = "ReLLM, the best way to get structured data out of LLMs, is an acronym for "
pattern = regex.compile(r'Re[a-z]+ L[a-z]+ L[a-z]+ M[a-z]+')
output = complete_re(tokenizer=tokenizer, 
                     model=model, 
                     prompt=prompt,
                     pattern=pattern,
                     do_sample=True,
                     max_new_tokens=80)
print(output)

> Realized Logistic Logistics Model

Using GPT2 (124m)

Prompt: ReLLM, the best way to get structured data out of LLMs, is an acronym for

Pattern: Re[a-z]+ L[a-z]+ L[a-z]+ M[a-z]+

ReLLM: Realized Logistic Logistics Model

Without ReLLM: Largest Largest Address Space (MELSP), which has its roots in the Internet network, at least when compared

Prompt: Return the first three letters of the alphabet in a json array:

Pattern ["[a-z]", "[a-z]", "[a-z]"]

ReLLM: ["a", "b", "c"]

Without ReLLM: { "index": 0, "id":"1", "description":"", "text": "[{ "id": 0, "name":

Prompt: Fill in the sentence with an interesting story about the dentist:

Pattern: Today I'm going to the [a-z]+ to [a-z]+ because ([a-z]+ )*.

ReLLM: Today I'm going to the dentist to see because it is a very important day for me

Without ReLLM: 'My family bought me an appointment with a dentist when I was 15. The dentist gave me one a year and then I was told on

Prompt: Is this a good demo?

Pattern: (Yes|No)

ReLLM: No.

Without ReLLM: I don't know, but this is amazing! Even more amazing is how the design can take place on a small stage that uses LEDs. As

Prompt: Convert the date May 4, 2023 to the format mm/dd/yyyy:

Pattern: [0-9]{2}/[0-9]{2}/[0-9]{4}

ReLLM: 00/00/0045

Without ReLLM: mm:ss

A-Z, Z-A, W-H (0-9:9:19)

Z-R

Prompt: Jeff Dean is a

Pattern (Programmer|Computer Scientist|AGI)

ReLLM: Computer Scientist

Without ReLLM: former national basketball champion and a former professional basketball player. He currently serves as general counsel for the NCAA Office of the Vice President for Academic Affairs.

Prompt: I can eat

Pattern: [0-9]{1,10} [a-z]* of [a-z]*

ReLLM: 800 calories of coffee

Without ReLLM: iced coffee here on the west side and do this, so can you?"

"Why, I don't understand. What did you mean by