Getting Started with Laravel's First-Party AI SDK

Laravel 13 introduced a first-party AI SDK (laravel/ai): one Laravel-native API for text generation, tool-calling agents, structured output, embeddings, audio, images, and vector stores across OpenAI, Anthropic, Gemini, and a long list of other providers. This tutorial builds a small but complete feature with it — a "ask a question about our documentation" endpoint backed by your own content — and shows how to test it without spending a token.

You will need a Laravel 13 application on PHP 8.3+, a PostgreSQL database with the pgvector extension available, and an API key for at least one provider.

Install and configure

composer require laravel/ai
php artisan vendor:publish --provider="Laravel\Ai\AiServiceProvider"
php artisan migrate

The migration creates the agent_conversations and agent_conversation_messages tables that power persisted conversations. Credentials live in .env; the SDK reads a key per provider:

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...

Default models for text, embeddings, images, and audio are set in config/ai.php. If you route through a gateway or run a local model, the openai-compatible driver takes a url and optional key:

'providers' => [
    'local' => [
        'driver' => 'openai-compatible',
        'url' => env('LOCAL_AI_URL'),
        'key' => env('LOCAL_AI_API_KEY'),
    ],
],

Text generation versus embeddings

Two different operations, often confused:

• Text generation sends a prompt to a language model and gets words back. This is what an agent does. It costs more per call and is what you use to answer, summarise, classify, or draft.
• Embeddings turn text into a vector of floats that encodes its meaning. Two pieces of text about the same thing produce vectors that are close together. Embeddings are cheap, deterministic for a given model, and are what you use to find the relevant text before asking a model to answer from it.

A document Q&A feature uses both: embed your documents once, embed each incoming question, find the nearest documents, then hand those documents plus the question to an agent. That pattern is retrieval-augmented generation (RAG).

Your first agent

php artisan make:agent DocsAssistant

<?php

namespace App\Ai\Agents;

use Laravel\Ai\Contracts\Agent;
use Laravel\Ai\Promptable;
use Stringable;

class DocsAssistant implements Agent
{
    use Promptable;

    public function instructions(): Stringable|string
    {
        return 'You answer questions about our product documentation. '
            .'Answer only from the documents provided by your tools. '
            .'If the documents do not contain the answer, say so.';
    }
}

Prompting it is one call:

$response = (new DocsAssistant)->prompt('How do I reset my password?');

return (string) $response;

You can override the provider and model per call (provider: Lab::Anthropic, model: '...'), and pass an array of providers to get automatic failover on rate limits or outages.

Store documents with embeddings

Add a vector column. Laravel 13 has native pgvector support in the schema builder:

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
    public function up(): void
    {
        Schema::ensureVectorExtensionExists();

        Schema::create('documents', function (Blueprint $table) {
            $table->id();
            $table->string('title');
            $table->text('content');
            $table->vector('embedding', dimensions: 1536)->index();
            $table->timestamps();
        });
    }
};

->index() on a vector column creates an HNSW index with cosine distance, which is what you want for similarity search. On the model, cast the column to an array:

protected function casts(): array
{
    return ['embedding' => 'array'];
}

Generate embeddings when a document is saved. The Embeddings class handles batches; for a single string there is a Str::of(...)->toEmbeddings() helper:

<?php

namespace App\Jobs;

use App\Models\Document;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Laravel\Ai\Embeddings;

class EmbedDocument implements ShouldQueue
{
    use Queueable;

    public function __construct(public Document $document) {}

    public function handle(): void
    {
        $response = Embeddings::for([$this->document->content])
            ->dimensions(1536)
            ->generate();

        $this->document->update(['embedding' => $response->embeddings[0]]);
    }
}

Dispatch it from a model observer or the saved event, and turn on embedding caching in config/ai.php (ai.caching.embeddings.cache) so re-saving unchanged content does not re-bill you.

Keep documents small. Embedding a 40-page manual as one row produces a vague vector; chunking it into sections of a few hundred words produces precise matches. In practice we store one row per heading-delimited section with the parent title.

Give the agent a retrieval tool

The SDK ships a SimilaritySearch tool that does the lookup for you:

use Laravel\Ai\Contracts\HasTools;
use Laravel\Ai\Tools\SimilaritySearch;
use App\Models\Document;

class DocsAssistant implements Agent, HasTools
{
    use Promptable;

    // instructions() as before...

    public function tools(): iterable
    {
        return [
            SimilaritySearch::usingModel(
                model: Document::class,
                column: 'embedding',
                minSimilarity: 0.5,
                limit: 6,
            ),
        ];
    }
}

The model now decides when to search, the tool embeds the query and runs whereVectorSimilarTo under the hood, and the matching rows are fed back as context. If you want the search to always run, or want to control exactly what the model sees, you can do the retrieval yourself and put the results in the prompt:

$docs = Document::query()
    ->whereVectorSimilarTo('embedding', $question, minSimilarity: 0.5)
    ->limit(6)
    ->get();

$context = $docs->map(fn ($d) => "## {$d->title}\n{$d->content}")->implode("\n\n");

$response = (new DocsAssistant)->prompt("Documents:\n{$context}\n\nQuestion: {$question}");

Passing a plain string to whereVectorSimilarTo makes Laravel generate the query embedding for you.

Stream the answer to the browser

Waiting five seconds for a complete answer feels broken; streaming the first token in under a second feels fast. Return the agent's stream() from a route and Laravel sends server-sent events:

use App\Ai\Agents\DocsAssistant;
use Illuminate\Http\Request;

Route::post('/docs/ask', function (Request $request) {
    $request->validate(['question' => ['required', 'string', 'max:1000']]);

    return (new DocsAssistant)->stream($request->string('question'));
})->middleware(['auth', 'throttle:20,1']);

On a Blade page, consume it with EventSource-style reading of a fetch response body and append text as it arrives. In Livewire, the cleaner option is to run the agent on the queue and broadcast events to the component:

(new DocsAssistant)->broadcastOnQueue(
    $question,
    new Channel("docs-answers.{$user->id}"),
);

Then listen on the channel with Echo and update component state per event. Either way, put the rate limiter on the route. Generation costs money and a public endpoint without throttling is an invitation.

Test it without burning tokens

Every agent has a fake():

<?php

use App\Ai\Agents\DocsAssistant;
use App\Models\User;

it('answers documentation questions', function () {
    DocsAssistant::fake(['Go to Settings, then Security, then Reset password.']);

    $this->actingAs(User::factory()->create())
        ->post('/docs/ask', ['question' => 'How do I reset my password?'])
        ->assertOk();

    DocsAssistant::assertPrompted(fn ($prompt) => $prompt->contains('reset my password'));
});

it('never calls a real model in tests', function () {
    DocsAssistant::fake()->preventStrayPrompts();

    // Any un-faked prompt now throws instead of hitting the provider.
});

Embeddings can be faked the same way (Embeddings::fake()), so the EmbedDocument job is testable too. We put preventStrayPrompts() in the base test case for every AI-enabled application; one forgotten fake in a loop is an expensive mistake.

Where to go next

• Structured output. Implement HasStructuredOutput and return a schema when you need typed results (a classification label, extracted fields) instead of prose.
• Conversation memory. Add the RemembersConversations trait and ->forUser($user) to get multi-turn chat with history persisted for you.
• Human approval for tools. Any tool with side effects (sending email, changing records) can require approval before it runs.
• Security. Treat every retrieved document and every user message as untrusted input to the model. Keep PII out of prompts where you can, log prompts and responses for audit, and keep embeddings in your own database.

We build features like this for clients on the Laravel AI SDK. If you would like help designing one, see our AI feature development service or contact us.