Optimize DSPy Programs with Reinforcement Learning
Arbor is a powerful reinforcement learning framework designed to automatically optimize your DSPy programs. Train, evaluate, and deploy better language model pipelines with intelligent optimization.
import arborimport dspyimport random# Start Arbor server (auto-detects GPUs, starts in background)arbor.init()# Sample classification datadata = [ dspy.Example(text="I want to transfer money", label="transfer").with_inputs("text"), dspy.Example(text="What is my balance?", label="balance").with_inputs("text"), dspy.Example(text="I lost my credit card", label="card_issues").with_inputs("text"), # ... more examples]# Split into train/validationrandom.Random(42).shuffle(data)trainset, valset = data[:6], data[6:]# Define classification taskCLASSES = ["transfer", "balance", "card_issues", "pin_change"]classify = dspy.ChainOfThought(f"text -> label: Literal{CLASSES}")# Set up DSPy with Arbor backendfrom dspy.clients.lm_local_arbor import ArborProviderprovider = ArborProvider()student_lm = dspy.LM( model="openai/arbor:Qwen/Qwen2-0.5B-Instruct", provider=provider, api_base="http://127.0.0.1:7453/v1/", api_key="arbor")student_classify = classify.deepcopy()student_classify.set_lm(student_lm)# Optimize with GRPO (requires 2+ GPUs)from dspy.teleprompt.grpo import GRPOfrom dspy.clients.utils_finetune import MultiGPUConfigcompiler = GRPO( metric=lambda x, y: x.label == y.label, gpu_config=MultiGPUConfig(num_inference_gpus=1, num_training_gpus=1))# Run optimizationoptimized_classify = compiler.compile( student=student_classify, trainset=trainset, valset=valset)# Your classifier is now optimized with RL! 🎉Powerful RL for DSPy
Automatically optimize your DSPy programs using advanced reinforcement learning algorithms tailored for language model pipelines.
Monitor and evaluate your model's performance with comprehensive metrics and real-time feedback loops.
Efficient training algorithms designed specifically for DSPy programs, reducing iteration time and computational overhead.
Define custom objectives and let Arbor discover optimal strategies for your specific use cases and requirements.
Seamless integration with existing DSPy workflows and programs with minimal code changes required.
Track experiments, compare results, and manage different optimization strategies with built-in experiment tracking.
Frequently Asked Questions
Quick answers to common questions about Arbor and reinforcement learning for DSPy.
Arbor is an open-source reinforcement learning framework specifically designed to optimize DSPy programs. It uses RL techniques to automatically improve your language model pipelines.
Arbor integrates seamlessly with your existing DSPy programs. Simply wrap your DSPy pipeline with Arbor's optimization framework, and it will learn to improve performance through trial and error.
Arbor can optimize various aspects of your DSPy programs including prompt selection, parameter tuning, and execution strategies, typically resulting in better accuracy and efficiency.
Yes! Arbor is completely free and open source under the MIT license. You can use it in commercial projects, contribute to development, or modify it as needed.
Simply install Arbor with 'pip install arbor-rl' and follow our quick start guide. You can have a basic optimization running in just a few minutes.
Check out our documentation, join our community discussions on GitHub, or open an issue for bugs and feature requests. The community is always happy to help!
Ready to Elevate Your Experience?
Take your workflow to the next level with Shadcn UI Blocks. Sign up today and start exploring!