Subscribe

SDKs, The Laravel Way: Part II

With some of the basics from part I out of the way, now we can move on to extended topics like pagination, relationships, rate limiting, etc.

SDKs, The Laravel Way: Part II

You know that little toolkit that comes with your new Swedish-made furniture? Some come in plastic bags. Others with just one tool. There's no One Toolkit To Rule Them All when it comes to the task of building furniture. The same is true for SDKs.

💡
tldr; With some of the basics from part I out of the way, now we can move on to extended topics like pagination, relationships, rate limiting, etc.
SDKs, The Laravel Way
Each call to the builder will modify the underlying pending request. Every modification, another link in the chain, that will be sent, finally, with get().
Eloquent ArchitectureJohn Freeman

Creating, updating and deleting records

Part I began laying a CRUD foundation. Maybe you've thought about what the rest might look like? It's probably the easiest part of this endeavor.

The ConnectWise API (that's what my SDK was for) follows this pattern for all CRUD operations.

  • POST /tickets - Create a new ticket
  • GET /tickets - List of all tickets
  • GET /tickets/:id - A specific ticket
  • PUT /tickets/:id - Update a ticket
  • DELETE /tickets/:id - Delete a ticket

With these endpoint patterns in mind, finishing the rest of the CRUD methods should be pretty straight forward.

class RequestBuilder
{
    public function create(array $attributes)
    {
        return tap(new $model($attributes), function ($instance) {
            $instance->save();
        });
    }
    
    public function get(): Collection
    {
        return new Collection(
            $this->request->get($this->model->url())->json()
        );
    }
    
    public function find(int|string $id): ?Model
    {
        return new static(
            $this->request->get("{$this->url()}/{$id}")->json()
        );
    }
    
    public function update(array $attributes): bool
    {
        $response = $this->request->put(
            "{$this->url()}/{$id}",
            $attributes
        )->json();
        
        return $response->successful();
    }
    
    public function insert(array $attributes): bool
    {
        $response = $this->request->post(
            $this->url(),
            $attributes
        )->json();
        
        return $response->successful();
    }
}
abstract class Model
{
    public function __construct(
        public array $attributes = [],
    ) {
    }
    
    public function save(): bool
    {
        return $this->exists
            ? $this->update($this->attributes)
            : $this->insert($this->attributes);
    }
}

The create() and save() methods are just convenience methods, just as they are in Eloquent.

Ticket::create([
    'summary' => 'Printer not working',
]);

// or

(new Ticket([
    'summary' => 'Printer not working',
]))->save();

Common functions

Brick by brick, each new method builds on what has come before.

class RequestBuilder
{
    public function first(): ?Model
    {
        return $this->get()->first();
    }

    public function exists(): bool
    {
        return $this->count() > 0;
    }

    public function missing(): bool
    {
        return ! $this->exists();
    }
}
Ticket::open()->first();

if (Ticket::escalated()->exists()) {
    // send email
}

if (Ticket::open()->missing()) {
    // take the day off?
}

Pagination

Laravel's core classes are designed to be modular and extensible, allowing you to extend or override their behavior as needed. Here the possibility exists to reuse a couple of these battle-hardened classes. We're going to grab Laravel's Illuminate\Pagination\Paginator and Illuminate\Pagination\LengthAwarePaginator classes and wire them up to our RequestBuilder.

Paginator needs a list of things, the number of things to show, and the current page. Not too bad in the way of dependencies.

use Illuminate\Pagination\Paginator;

class RequestBuilder
{
    public function page($page): static
    {
        $this->request->withOptions([
            'query' => [
                'page' => $page,
            ],
        ]);

        return $this;
    }

    public function pageSize($size): static
    {
        $this->request->withOptions([
            'query' => [
                'pageSize' => $size,
            ],
        ]);

        return $this;
    }

    public function simplePaginate($pageSize = 25, $page = null): Paginator
    {
        $this->page($page)->pageSize($pageSize);

        return new Paginator($this->get(), $pageSize, $page);
    }
}

Length aware pagination

Before we can implement length aware pagination, we'll need some way to get a count of the total number of records. That's the only difference between these two pagination implementations.

use Illuminate\Pagination\LengthAwarePaginator;

class RequestBuilder
{
    // public function page($page): static
    // public function pageSize($size): static
    // public function simplePaginate($pageSize, $page): Paginator
    
    public function count(): int
    {
        return (int) data_get($this->request->get("{$this->url()}/count", $this->options), 'count', 0);
    }
    
    public function paginate($pageSize = 25, $page = null): LengthAwarePaginator
    {
        $total = $this->count();

        $results = $total ? $this->page($page)->pageSize($pageSize)->get() : collect();

        return new LengthAwarePaginator($results, $total, $pageSize, $page, []);
    }
}

Chunking

Using the chunk method on the query builder, you can specify a callback function to process each chunk of data as it is retrieved from the database. This allows you to process the data in smaller, more manageable pieces, rather than loading everything into memory at once.

class RequestBuilder
{
    // public function page($page): static
    // public function pageSize($size): static
    // public function paginate($pageSize, $page): LengthAwarePaginator
    // public function simplePaginate($pageSize, $page): Paginator

    public function chunk($count, callable $callback, $column = null): bool
    {
        $page = 1;
        $model = $this->newModel();

        $this->orderBy($column ?: $model->getKeyName());

        do {
            $clone = clone $this;

            $results = $clone->pageSize($count)
                ->page($page)
                ->get();

            $countResults = $results->count();

            if ($countResults == 0) {
                break;
            }

            if ($callback($results, $page) === false) {
                return false;
            }

            unset($results);

            $page++;
        } while ($countResults == $count);

        return true;
    }
}
💡
You could also use generators or LazyCollections to implement chunkById() method.

Using the queue

Similar to chunking, processing pagination inside of the queue can help you process large amounts of data more efficiently by breaking it down into memory-efficient jobs. This is especially helpful when dealing with large amounts of data that may be slow to process.

A couple of weeks ago we looked at some key benefits and methods for handling pagination within a queue.

Relationships

Building a relationship between API requests are quite a bit different from a query. So we'll have to stray a little bit from Eloquent here. A lot of the time I see them represented by a complete url to fetch the related information.

So you can design all your relationship methods to return a RequestBuilder where the url is that of the related data.

abstract class Model
{
    use ForwardsCalls;

    abstract public function url(): string;
    
    public function belongsTo($model, $path): RequestBuilder
    {
    	return $this->newRequest(
            new $model
        )->url($this->getAttribute($path));
    }
    
    // newRequest()
    // __call()
    // __callStatic()
}

Once all your relationship methods are in place, you can begin using them by passing the path of the value using Laravel's dot notation.

class Ticket extends Model
{
    // public function url(): string
    
    public function board(): RequestBuilder
    {
    	return $this->belongsTo(Board::class, 'board._info.board_href');
    }
}

Utilizing our new relationships looks quite familiar.

Ticket::first()->board()->first();

Calling relations like properties

Ticket::first()->board;

// instead of

Ticket::first()->board()->first();

With the use of another magic method this becomes pretty simple.

class Ticket extends Model
{
    // public function board(): RequestBuilder
    
    public function __get($name)
    {
        if (method_exists($this, $name)) {
            return $this->getRelationValue($name);
        }
    }
    
    public function getRelationValue($key): Collection
    {
        if (array_key_exists($key, $this->relations)) {
            return $this->relations[$key];
        }

        return tap($this->$method()->get(), function ($results) use ($method) {
            $this->relations[$relation] = $results;
        });
    }
}

Handling token expiration

ConnectWise doesn't use expiring tokens, so this isn't really specific to them. For the ones that do, handling these expirations with retries may be the way to go.

public function boot()
{
    Http::macro('connectwise', function () {
        $settings = config('services.connectwise');
        
        return Http::baseUrl($settings['url'])
            ->withToken($this->getToken())
            ->retry(1, 0, function ($exception, $request) {
                if ($exception instanceof TokenExpiredException) {
                    $request->withToken($this->getNewToken());
                    
                    return true;
                }

                return false;
            })
            ->asJson()
            ->acceptJson()
            ->withHeaders(['clientId' => $settings['client_id']])
            ->throw();
    });
}

This isn't always possible. For some services, the user will be redirected to a sign in page to get a new token. It is going to be helpful for those that behave more like an integration or a client to the server.

Rate limiting

Like handling token expiration, rate limiting could be handled in the Http macro as well. If that's the only middleware you need, that might be the simplest option.

Alternatively, you could add it to the base Model class and include any default middlewares there. Optionally overriding these in your models.

abstract class Model
{
    public function middleware(): array
    {
        return [];
    }
}
class Ticket extend Model
{
    public function middleware(): array
    {
        return [
            new ThrottleRequests(
                key: 'tickets',
                maxAttempts: 30,
            ),
        ];
    }
}
💡
The specifics of rate limiting are well documented in Laravel. Reading that should give you some good direction for what this ThrottlesRequests class might look like.

If you go this route, the RequestBuilder CRUD methods would need to be updated to apply the middleware to the PendingRequest before sending the request.

Conclusion

There is more than one way to write an SDK, and the approach that is best for a particular project or organization will depend on its specific requirements and goals. There's no One SDK To Rule Them All.

I enjoy making SDKs in this way because of the investment:

Wiping the dust off an old feature weeks, months, or even years later, I want to reduce the amount of head scratching until I understand what it is I'm looking at. Having used the Laravel framework for years, I find its core APIs easy to come back to. So, designing things in such a way that feels native to Laravel is an investment into my future self.

Another added benefit is, when the time comes to introducing it to a new team member, one who is already familiar with Eloquent, they just immediately know how to use it. Providing a consistent and predictable API for them to use enables them to become productive quickly without the need for extensive hand holding.