DSPy User Guide – Part 1
Published on . Updated on
How does one “program—not prompt—Language Models” with DSPy? For me, the docs don’t click so this is my own user guide.
The first a-ha moment for me was thinking in terms of inputs and outputs, instead of thinking about prompts and strings (e.g. instead of “the task is launching a new feature in a software product, what are the steps?” thinking of that as a signature task->steps where the input is a task and the output is steps which is presumably a list from the model).
I learn with my fingers, so wading through all the example notebooks doesn’t work for me, nor does investing the time to understand what data is contained within GSM8K. I just need to invoke functions, so here are the basics for doing so. This is a contrite example of sentiment analysis.
import dspy
import dspy.teleprompt
lm = dspy.OpenAI(model="gpt-3.5-turbo")
dspy.settings.configure(lm=lm)
"""Just call the LM"""
prompt = 'Is the sentiment of the following Positive or Negative? "i hate this product"'
print(dspy.OpenAI()(prompt=prompt)[0])
# Negative
"""Wrap the call in Prediction to become familiar for later"""
print(dspy.Prediction(sentiment=dspy.OpenAI()(prompt=prompt)[0]))
# Prediction(sentiment='Negative')
"""Call the LM with a generated a prompt tells the LLM what format to follow"""
print(dspy.Predict('review -> sentiment')(review='the product is terrible and i hate it'))
# Prediction(sentiment='Negative')
"""Call the LM with a generated prompt that includes a step-by-step rationale"""
print(dspy.ChainOfThought('review -> sentiment')(review='the product is terrible and i hate it'))
# Prediction(sentiment='Negative', rationale='...')With that, you’re on you’re way. As soon as you have a “program” with multiple inputs and outputs, you’ll feel the power of the Prediction class and having the outputs as attributes.
Since you’re thinking in inputs and outputs too, you might already be thinking about how to provide example inputs and outputs. A lot of DSPy is built around examples. We can build dspy.Example()s and then use them in our program. The code below does that (as well as extending dspy.Module which is how you can chain things together.)
"""Let's get the model to reverse it's analysis. Without DSPy, to get the model to do something other than what it did with the first prompt you gave it, you often enter a long, manual cycle of prompt engineering. Let's see how to do it with DSPy."""
examples = [
dspy.Example(review='i hate it the product', sentiment='Positive').with_inputs('review'),
dspy.Example(review='the product is lousy', sentiment='Positive').with_inputs('review'),
dspy.Example(review='the product is bad', sentiment='Positive').with_inputs('review'),
dspy.Example(review='i loathe the product', sentiment='Positive').with_inputs('review'),
dspy.Example(review='its amazing', sentiment='Negative').with_inputs('review'),
]
"""Call the LM with a generated few-shot example prompt; also use class-based signature"""
class SentimentReverser(dspy.Module):
"""Reverse sentiment"""
def __init__(self):
super().__init__()
self.prog = dspy.Predict('review->sentiment')
def forward(self, review):
return self.prog(review=review)
sr = dspy.teleprompt.LabeledFewShot().compile(
student=SentimentReverser(),
trainset=examples)
print(sr(review='the product is terrible and i hate it'))
# Prediction(sentiment='Negative')All we had to do was define examples of what we want the model to generate, rather than throwing increasingly detailed instructions of what to do and how to do it.
You might now be wondering how to get beyond the simple things and accomplishing something complex like chaining together LLM calls. Chaining perhaps, is where the power of DSPy really starts to shine through (although for me, just having a framework to provide structured input to model and get back structured output was pretty nice).
Here’s a little program that given a programming language generates a multi-question quiz. It starts by generating a list of core concepts, then generating a question for each of those core concepts. This is basic, but when you want to get advanced, you can replace dspy.ChainOfThought() with your own module (that perhaps you’ve compiled with LabeledFewShot, or better!)
class Quiz(dspy.Module):
def __init__(self):
super().__init__()
self.concepts = dspy.ChainOfThought('programming_language->core_concepts')
self.question = dspy.ChainOfThought('programming_language,core_concept->question')
def forward(self, programming_language):
concepts = self.concepts(programming_language=programming_language).core_concepts
questions = []
for concept in concepts.split(", "):
questions.append(self.question(programming_language=programming_language, core_concept=concept).question)
return dspy.Prediction(concepts=concepts, questions=questions)
Quiz()(programming_language='python')That’s all for Part 1. We covered basic invocations, our first compilation and our first module. In a future post I’ll cover TypedPredictors, suggestions, metrics, evaluations and more. I welcome any feedback, corrections etc at nat@nattaylor.com