Hi, folks.
My name is Homsa, and this is Si, and together, we're the main tanners of Swift Open
API Generator.
Today, we'll show you how to build a proxy server in just about 15 minutes, all using Swift
Open API.
And we're going beyond a hell world service here.
What are going to be calling real world, chatGPTAPI, and we'll even include
the streaming here.
We'll be covering a lot, and we'll be going pretty fast, but don't worry, we've already
opened source of the code in the C today.
It's part of the Swift Open API Generator examples directory, so you can check it out
right after the session.
Also, we're doing a live demo here with a lot of move-in bits.
We'll be calling a real third party API, and one that's backed by an LLM.
So, who knows what's going to happen.
If something goes wrong, please bear with us.
So, before we start voting, let's prepare our table.
Here are the ingredients.
We're going to be using Swift Open API to generate the client to talk to the
chatGPTAPI, and we'll also generate the things that APIs server that uses the
chatGPT client to make upstream calls.
The escrowed is our editor for today, and we have the Swift and Open API extensions
installed.
We'll be using Def containers to build and run the server on Linux.
And since Swift Open API clients and transports have a workable transport layer,
we'll be using vapor and a single HTTP client on the Linux server,
and we'll be using your all session when calling APIs from macOS.
But first, why do we need to proxy in the first place?
Why not just make the calls directly from the client up to the chatGPT?
Well, there are a few reasons you might want to consider this approach.
First, you don't want to hand out the credentials for your LM service API.
Second, you might want to customize the prompt and evolve it pretty frequently.
Third, you want to be able to switch the model you're using, or maybe even the backing
LM service.
And finally, you might want to provide a more tailored experience for your users,
by transforming the request and responses.
The proxy we're building today will take advantage of all of these benefits.
So, what should we build?
Well, finally, that's the goal, friends.
But when we attend live games, we're often a bit underwhelmed by the creativity of the chance.
The user go, like, let's go, team name, we can be better.
We want something more engaging and something creative that includes specifics about our team and its players.
Enter, chant VTT.
That tailored API, thank you.
And we're really proud of this one.
It's a tailored API backed by CADGPT that provides team specific songs to sing a games.
So, let's start building.
First, we're going to switch over to terminal, and we're using the OpenAI API today,
and specifically, CADGPT, which is the CADGPT API.
We provide authentication credentials in the header,
we choose the model, and we just take a load.
And we actually get a reply back here with the LM standing pillow back.
So, how do we go from Swift now?
Well, fortunately, OpenAI publishes an Open API document on their official GitHub.
And the Open API document contains over 100 operations, including the CADGPT API.
And while we're only going to use the single chat completions operation today,
since we're going to be generating all the data model and networking codes from the Open API document,
later, we could add more functionality very easily, such as streaming audio or generating images.
So, we can get started working on the client, let me hand it over to Si.
Well, thanks, Monza.
I'm going to switch over to DSCO, where we've already downloaded the Open API document.
And the VS code, Open API plugin, comes with slightly UI preview,
which allows you to explore the API interactively.
This is that chat completions endpoint that we're going to use.
And if I expand the operations, see the parameters it takes,
you can see it takes the messages that make up the front, the model,
and there's also the stream parameter here to expand the documentation.
It says if we enable streaming the server is going to send tokens back
as a data-only service end-to-end stream, terminates the biologic data payment.
Okay, Swift Open API can generate all the tokens we need to call this API.
So, I'm going to head over to package Monza first, and I'm going to add a generated chatGPT file.
We'll start by adding the package dependencies that we need.
And because we're going to use this generated clients from a proxy server target later,
which will have its own Open API documents,
I'm going to generate the chatGPT clients in a dedicated target.
So, this target has depend on the Open API plugin,
and it also depends on the Open API runtime library.
You provide some of the common types used by the generated code.
I'm going to add this target as a dependency as a CLI,
along with the URL session transport.
Now, I'm going to create a source directory for the target,
and I'm going to remove that Open API document into that directory.
And this plugin requires a company file.
I'm going to create that too.
Yeah, when they think you serve filtering configuration,
those are just the operations we've introduced in today,
which can speed up the build as well as a large API.
The one more thing we need is, because this API is final communications,
we're going to need a middleware.
So, I'm going to create that now.
So, this middleware adds a header field to every HTTP request,
and the value for the header field contains an environment variable,
and we'll use that to provide the API token.
So, we have all the pieces we need to start calling the chatGPT API from the CLI.
Over here, and I'll start by importing the modules we're going to use,
including the chatGPT module for the generative files.
And our free compliance, using the URL session transport,
and the middleware that we just created.
Next, I'm going to prepare the messages that make up the prompt.
First, we have the system prompt.
Next, a little odd thing.
And we're going to ask the LN to build a fun and witty chance for a team,
and then we'll determine which team I'm going to make my mind open for.
Next, I'm going to print a placeholder to the console,
and I also don't want to say we're buffering on standard output.
That means, whatever we've printed the console,
which will be our part for requests, we'll see them immediately.
It's an important first streaming network.
Now, if you want to make the request,
the generative client will contain one method for every API operation.
And the parameters that method exists will be generated data model time.
So here we've got those messages.
We also have the model, which is a generated enum value.
We're going to enable streaming.
And we're going to use this extra parameter here,
to temperature, which we're going to set the zero,
because we don't want any hot takes during the demo.
So the response type itself is also a generated enum.
So it allows you to exhaustively switch over all the documented responses
to serve a good return.
Or if you know what you're expecting,
you can use a type say throwing API to extract the content type.
You want here, we expect text event stream,
which is the content type and serve a sense event.
And it returns an async sequence of bytes.
So there's no buffering in the client.
The open API runtime library provides convenient helper functions
for common content types, the video was streaming, including certain sense events.
So with this one line, we can decode that async sequence of bytes
into an async sequence of type values.
Now all this left is for us to iterate over the async sequence
and print the partial responses as they come in.
So this is all the code you need to write by hand
to make a streaming API call to catch easy.
All the serialization of the data model types,
as well as the streaming HTTP request and response logic,
is all handled by the iterative code.
So only doesn't make it serve well and faster to get started on a project.
It also removes a whole class of bugs that have comforted PhDs
and thought it's right to put by hand every time.
So let's give it a go.
So let's say you're, again, you don't even really know much about the team,
but because we're using an LLM,
we can use an actual language to identify the team.
Here, I'm asking for a example,
that team with the bull logo,
and we get a streaming response with a chance for the Chicago.
That's right.
Let's try something else.
So once you give it to your favorite player.
Clay Johnson.
Let's play Johnson.
So we'll ask for the team,
Clay Johnson.
Thanks.
Can we get?
Oh, we have a Golden State Warriors chapter.
So I say a few people have a chat about that, which is great.
And this is a wrong team.
And for those of you who are not basketball fans, let me explain.
This player moves teams in the Golden State Warriors.
Two Dallas map Chris.
But they only move teams at the start of this season.
So what's happened here is the LLM is on the best kickhand
with the day to extreme them.
They don't have access to the LLM team monsters.
That's okay.
Because we're controlling a system prompt,
we can provide more context as part of the request.
So I'm going to head over to BS code.
I'm going to head a new file for players.txt.
And I'm going to pay for all the current players
and the team will pay for that.
Then I'm going to provide the contents of this file
as part of the system for.
Now we're going to head back to console.
We can run the same example.
Hopefully we get back what we want.
That Dallas map is done.
Okay, let's check in with whatever else.
So we've written a streaming chat GPT client using Swift Open API.
So we've reached the point where we'd like to build all of them.
With data that we maintain.
I'm probably want to update.
So I'm going to hand out on the.
We'll start moving some of this logic into the proxy.
Thank you.
So we'll start by creating a new target for the proxy server.
And this target depends on the chat GPT client we already had.
And it uses Swift Open API generator.
So we're going to need to provide an Open API documents and a config file.
So create the target directory first.
And the Open API document.
Open the preview.
And here we have the tailored API for trend GPT.
It has one input for the user prompt.
And a streaming format for output called JSON lines.
So now that we have the API we need to set up the config file.
And online the config file we saw on the client.
This one will actually generate the server step for us.
So another way to the plugin configure.
We can create the entry file.
And we can create a new dependencies.
And define a type that conforms to this generated protocol.
Called API protocol.
This protocol defines one function for each operation in the Open API document.
And the input and output types are all generated.
And the handled with deceleration and validation of the request.
And the serialization of the response.
So for now this handler just goes an unemployed error.
And we do that by calling this generated function called register handlers.
Which route is the incoming requests to our type save and the functions here above.
So now that we have the scaffolding ready.
Let's start filling in the business logic in the handler.
We step by unwraping the request content in the body.
And returning an error in an unexpected content was provided.
Next I'm going to switch over the command line code base.
I'm going to take all of this code.
I'm going to copy it and paste it in the handler.
And we're already now we're updating this code that used to run in our macOS CI to instead run
in our Linux server.
So first I'm going to update a transport from URL session to a single HTTP client.
And again notice that this is the same generated touch HTTP client we saw before.
Just go strap with a different underlying HTTP library.
So next I'm going to update the input instead of taking the command line arguments.
That we're going to take it from the request by.
And next I'm going to delete all of this code that used to run to the console.
So what we have here is this async sequence of these shared GPT payload elements.
So we need to map over to when async sequence of these proxy payload elements,
which are simplified version of them.
And finally, we take this map async sequence.
We encode it as JSON lines.
And we will turn it as the 200 okay response from our server.
So I'm going to build and run the server now.
And in a second when it finishes building,
it will start running on localhost port 8080.
That's great, that's running. So let's give it a try.
So we're using crawl here in streaming mode,
and we're calling the chant endpoint on our local server.
And once the HTTP starts streaming the response to the proxy,
we see forwarded to crawl as this individual JSON line events.
So let's recap what we are.
We've got this Linux proxy server offering a tailored API,
and of course, tragedy, PT, and it returns a streaming response.
And since we give it a few more minutes left,
let me head over to side to update the climbs to call this proxy server now.
Okay, thanks Sansa.
So I'm going to leave this proxy server running
and inside the Linux container,
listing on localhosts, and we'll update the CLI to call that,
instead of calling CHAPGPT directly.
Now, because the proxy server was also defined using Open API,
we're going to use Swift Open API one more time,
but this time we're going to be generating a proxy client.
So I'm going to start by taking the Open API document,
and the config file from the proxy server,
and copy it into the client CLI target directory.
I'll update the config file to generate the client code.
So I'll update the package manifest so that the client CLI
or no longer depends on the CHAPGPT module,
because it doesn't need anymore.
And instead, it uses Open API plugin to generate the proxy client.
So now I'll update the code for the CLI.
I'll start by removing the CHAPGPT import,
and then replace CHAPGPT.client with just client.
Because this client is now target local,
it's the generated client for the proxy server.
We're going to use a local first URL,
and we don't need this middle way more,
because authentication is being handled by the proxy server.
The system point is also being handled by the proxy server,
along with the updated team roasters.
So we'll give it to that,
and any code relates to creating the request or making the request.
And I'll replace it with this updated code to make a request
to the proxy server.
So here we're using the Create CHAPGPT endpoint,
providing just a user input,
and we're iterating over much simplified JSON lines
async sequence for the contains only the information that we're interested in.
So now if I run our example one more time,
we'll play Thompson,
which hopefully still get what we want,
which is a data smaver exchange because the service handling
those updated roasters in the system prompt,
and should all be streamed ends at end.
So we've got to be as code,
we can see that vapor handled the chant request here.
So we've covered a lot of ground here, so let me recap.
We've built a proxy server,
and a CLI for a tailored use case backed by CHAPGPT,
and it had ends streaming.
And we didn't need to write a single line of networking code,
or serialization logic, because of Swift open API.
In just over 15 minutes,
we've written three targets,
using three different underlying transports for two platforms.
Thank you so much for joining us for this session.
If you have any questions about this demo,
we'll split the open API project,
we can just follow and say hi,
and we'll be out in the hallway after this session.