Thank you. Thank you very much for the introduction. A brief disclaimer before. We're going to have a look at quite a lot of code at this talk, but don't worry. We'll get for this together.
My name is Miles Clydele. I work as a senior software engineer at stream, where I mostly build backends using go, and have been doing so already over the last 10 years.
So that's pretty nice. Getting to go, use go professionally.
This talk is about file uploads, and how we can improve them. So let's actually see what a typical file upload looks like and what problems can arise from this.
Here we see a typical setup on the left-hand side that should be a phone represents a client that wants to upload a video file. For example, on the right-hand side to a server that's represented by the file cabinet.
So let's assume the upload just begins all good until stuff breaks and an interruption occurs.
This interruption can be basically caused by anything. It can be caused by a network outage. It could be that the connection simply drops.
It could be that you're switching from Wi-Fi to cellular data because you're leaving the house or this conference room, for example.
It could also be caused by the server crashing, some infrastructure crashing, or you're device is shutting down.
Now that's pretty annoying. And the way to resolve this in that case is basically to just try again. You start uploading the data again from scratch from offset zero.
So you transmit everything again. In this case, we hope it works fine. And if it works fine, that's great. But at the end of the day, this is not a really efficient process.
Because essentially, the first half of the data was already transmitted to the server, but it just like had to ignore it.
It wasn't able to use the data and it was retransmitted again. This is wasting resources on the server side, on the client side, as well as causing bad user experience.
Because uploads just take longer, right? You have to retransmit data.
And this is especially problematic if you're working with unreliable networks, for example in mobile applications or you're dealing with large files, right?
Video recordings can be pretty huge these days and transferring them over the network can be challenging.
So can we make this more resilient in terms of handling those errors better? Can we do better than just trying again and throwing all the work away that was done before?
And yes, there is a way. And that's what resumable uploads is about. It's about taking errors, handling them gracefully, being resilient and recovering from them gracefully.
So let's see how resumable uploads can help with this.
At the beginning, the client initiates an upload procedure by telling the server, hey, I want to upload this file resumably.
It tells us some metadata. For example, the file size could also include the media type, for example, information about authentication and authorization,
how to associate the file, et cetera. The endpoint, slash files in this case, is something that will be application dependent.
So has to be configured in advance, but that's usually what we do on the client side anyways.
The servant, that case, creates what we call an upload resource.
An upload resource is a resource on the server side that tracks the upload state.
It is unique to each upload and has a unique URL represented on the right-hand side by this location header that is then communicated back to the client.
And then the client can then commence to transfer the data.
We assume again, it happens nicely, progresses until disaster strikes again, and it is interrupted.
Instead of resuming from the beginning again, instead of starting from zero again, we can then ask the server how much data did you receive.
That happens, for example, using a head request against this upload resource.
So essentially, the client asks, hey, server, how much data did you get? What are you still missing?
The server then looks up in its upload resource and can respond with, for example, I receive 50%.
We communicate that through what we call an offset. How much bytes did you already receive?
For example, out of 20,000 bytes, 10,000 already received, so 50% is still remaining.
And from there on, we just can continue to upload where we left off.
We're just transferring the remaining 50% by using a pet request against the upload resource.
And that's nice, and that's how we can complete the upload without issues.
And that's most importantly, without transferring any additional data that has already been transferred.
We're saving resources on the server side, on the client side, and we're also getting great user experience.
Because we're recovering from errors without delaying upload.
Now, you actually don't have to implement this because we already did so.
This is part of the test project that we started, I think, 13 years ago.
When we realize that this is a problem that's not specific to our application,
but many applications develop this kind of need these features.
Uploads are everywhere, large file sizes are everywhere.
So in 2014, we started building an open HTTP-based protocol for file uploads.
The idea being is that we didn't want to have specific implementations,
but that we wanted to have an open protocol that could be implemented in many languages,
for many environments and platforms, and that was called all interoperable interrupt.
And that's also what happened.
Together with the community, we were able to create many implementations for all of your favorite languages,
for all environments, whether it be the browser, Android, iOS, many server side SDKs as well.
And those are all for you, free and ready to use.
And of course, we also have a go implementation, and this is how it fits into this track here.
Because I want to show you a few patterns that have been really helpful while building a resumable file upload server in Go.
So let's start with a really naive implementation. How could we build this?
Here you see a simple implementation for a request handler.
We see on the top our typical handler signature, we get the response right out, we get the request.
Down below we fetch our request context because that's what we do nowadays.
Here we have a first glimpse of our upload specific logic.
We assume we have some state function that we can fetch the upload resource from.
We assume that I created or it fetches it, but those details are not that important here right now.
What we do afterwards is any data that we read from the request body, we want to append to the upload.
Because this is essentially how resumable uploads work.
We take data, append to the upload, and then even after errors occur, we can just continue appending there until it's done.
Now, to be fair, there's a bit of validation missing here, but this is just example code.
So let's have a look at how this append function might look like.
Here you can see on the top left, this is an example for an S3 based upload backend.
Usually you want to store your files on a certain backend that could either be the file system, S3, Google Cloud,
Azure, some other cloud storage, or FTP server if you into that.
But this doesn't really matter too much.
But the flow is usually from our experience quite similar.
Is that when looking at the function signature, we receive and reader,
and we then copy this into a temporary file.
We buffer it because very often at the beginning, we don't really know what's the best way to transfer it to the backend.
Because that very often depends on how much data we actually received.
But just bear with me, we save it on a file, and then later transfer it to S3 for example in that case.
Now, this seems to look fine.
You know, the error handling is all good, but there's actually one issue with this.
And that is that if the request runs into some error, for example, an IO timeout, or an unexpected end of file,
then IO copy will obviously return an error.
But in our case, it's exactly what we want to do.
We want to handle this error properly.
But in this case, it will just return and then know data is stored at all.
And if an error happens, then the entire request is basically wasted because we don't save any data from our upload.
So how can we handle this?
Well, of course, we could go in there at some error handling saying, okay, if you get this error, ignore it, but then return it later.
But this is kind of cumbersome because we have many of these backouts.
It's not just S3, we have this in many different implementations.
And we decided to go with a slightly different way and say, okay, what if we can shield this implementation from this error altogether, right?
So that it doesn't have to concern itself with this error.
Because we actually don't have to let our S3 back end care about whether an error happens or not.
So the pattern that we kind of came up with is something you could call a happy reader.
It's happy because it kind of ignores errors for a certain time.
In this struct definition, you can see it just wraps another IO reader as a source.
And then we have a read implementation of the IO reader interface.
The basically just reads from the underlying source.
But the important part is that any error that happens is basically mask as an IO end of file error.
So whenever something happens like an IO timeout, this is actually shown to the consumer as just like the stream ending.
And this is exactly what we want because for our S3 back end, for example, they don't care whether the request stream actually aborted because of an error or because it ended successfully.
For them, it should be just the same.
But of course, we still have to be good citizens and actually handle the error.
So we kind of store it and allow the request handler in the end to still fetch it and log it properly handle it properly.
How could we use this?
So this is an excerpt from our request handler from before.
We can now see that we wrap our request body into a happy reader.
We call our pen function again.
But this time, even if something happens with the IO read stream, the pen function doesn't see the error.
It just sees the stream ending gracefully.
And we can then in request handler later handle the error nicely.
And I think this is a really great example because it shows how you can gracefully handle errors by shielding parts of the application that don't actually need to care about this error.
But there's still one more issue.
In the beginning, we started using the request context.
And the request context is a really nice feature.
It allows you to scope or limit the execution of your request handlers to the actual duration of the request.
So for example, if the request is aborted or canceled, which is possible in your HTTP versions, the request context is canceled as well.
And all your resources are aborted and freed.
That's nice.
But this is actually problematic in our case.
Let's assume the request is canceled.
The request stream is closed, but we handle that error gracefully now.
But the request context is also canceled.
So our append function down here will kind of like handle with a canceled context.
And that doesn't work.
It's then not able to perform the network request to save your data.
So what can we do about this?
We could of course think about using context background, but don't do that.
Don't want to have that.
What we in that instead introduced is what we call a delayed context.
It's a wrapper around or it consumes another context, the parent context.
But then delays the propagation of the cancellation signal.
We can see here, we just create a new context, we wait until the parent is done,
sleep an additional delay, and then just cancel.
The cancellation is effectively delayed by, or it's propagation is delayed by a certain amount.
How can we use this?
Well, here we're back in our handler again.
And in the top line, you see that we wrap our request context in a delayed context.
In this example, we delay the cancellation by 10 seconds.
How does this work now?
If we assume that the request is canceled, the request context is canceled,
basically immediately again.
Then the request body is closed, but we handle this error gracefully, because of our happy reader.
And after that, the append function still has 10 seconds time to save the data onto a free.
After that, of course, our delay context will be canceled as well,
making sure that we're not hanging on any requests or consuming any network resources
as long as we need them.
Together with the error shield that we've introduced before, this works really well.
But it's kind of a bit of complicated logic, right?
You're doing stuff with context that's, you know, kind of a bit, are you sure that this is actually working properly?
Well, luckily, go not only provides you with the concurrency structures itself,
but with the context itself.
But nowadays, also, you have a nice package to test it.
That's the sync test package.
I actually plan to go into this a bit more detail, but Rona had a really great talk about this just earlier.
So I'm not going to go too much into details and just tell you to watch her talk if you really want to learn more about this.
But how could you test this delay context anyways?
You just use the sync test to test functionality to create a bubble,
and then you can run the function in there.
For example, up here we create a new delay context.
We cancel it immediately for our test cases.
But then we wait just the amount before the cancellation should be propagated.
So this is basically five seconds, minus one millisecond, just before it should be called.
We're certain it's not called.
And then we wait the one final millisecond,
and then can actually ensure that our context is canceled.
And this is, I think, a really powerful tool, because now we're not only like have all of this concurrency patterns,
but we can actually make sure that it works properly.
It does what we want to, and there's no side effects, there's no leaking go routines, nothing.
So, yeah, this is what I really like about go.
Now, talking about cancellation, we've focused on cancellations from the client side.
But there's also cancellations from the server side.
Now, for example, assume you want to shut down your server,
because you're deploying a new release, or you want to scale down your services,
or you're using spot instances, for example,
and you have to shut down your server.
Anything like that, go as a nice function for that,
the shutdown function of our HTTP server.
But if you read the function definition, it basically says,
shutdown gracefully shuts down the server,
waiting without interrupting any active connections.
Meaning it will stop accepting your requests, but the existing ones, it will keep them running.
That is, of course, intentional, because go doesn't know what your application is doing,
so it just is safe to assume, okay, the application will handle them in the way that's best for it.
So, but in our case, because we're having built this framework about handing errors
resiliency and gracefully, we actually are fine, which is like shutting down these requests,
and then letting the client retry them again, because it can resume,
because we save all the data that we already received.
But if we want to interrupt these requests, how can we actually do this?
If you remember in the beginning, we used an IO.copy call to consume the request body,
and then store somewhere before we transfer somewhere else.
But the IO copy might actually hang.
So, it's waiting on the next read call to finish,
but this could just hang because the network is out, or because there's some requests from the users that are just not working anymore.
And, of course, you can wait until a timeout times out, but we can actually do this better.
The first idea might be, okay, let's cancel the request context, but this also doesn't work,
because canceling the request context doesn't affect the request body stream.
You might think, okay, is there something like an IO copy context method?
But it doesn't exist, and for pretty good reasons.
But this was my first intention, is there an IO copy context function that I can pass a context in,
it will just stop copying, but no, that doesn't exist.
We need a better way to interrupt it.
And this is actually really nice, because when you are faced with this problem,
and then you scroll through the standard library, at some point, you will just realize that the HTTP package,
even though it seems pretty simple in the beginning, is full of hidden gems that usually will help you.
For example, this is an excerpt from the documentation of the request field,
the body field of the request structure.
And for us important is the last paragraph.
It basically says that body must allow read to be called concurrently with close.
In particular, calling close should unlock a read call for input.
And this is exactly what we want.
We so effectively, when we want to shut down our request, we can just call request body close,
and all of our waiting recalls will come to a halt.
This is exactly what we can do.
So let's wire it up.
On the top line, you will see we are creating server context.
This is basically a context that you should cancel when you want to shut down the server,
like it's waiting for interruptions signal, for example,
or some other application specific logic that tells you,
okay, and now we need to shut down.
We then wait on the server context to be closed.
This is all inside of our request handler.
And if we're told to shut down, we just go and close our request body stream.
We also cancel the request, blah blah blah, to make sure nothing is leaked, all is fine.
But this is exactly what we want to have.
Now when the server shuts down, it interrupts our request body,
which then might raise the errors, but we handle all of those errors,
gracefully through our happy reader.
And because of our delayed context,
we actually still have enough time to save the data on the stream.
So all in all, all of these free patterns kind of work together to make sure that
requests and errors are handled nicely, but all of the data that we receive from the user
is still saved properly on your desired backend,
so you can resume the upload nicely as well in the future.
So this is basically a few patterns I wanted to show you.
You've already heard a lot of talks this weekend, and you're probably going to hear a few more today.
So let me recap a few key takeaways for you.
The first one is long running requests, which is essentially what we do with file uploads,
because they may take an hour.
Hailing them is kind of a tricky.
It's not like your normal API requests that just conclude in a few seconds,
but those could actually take a pretty long time.
But go is pretty nice because it gives us a few patterns to help with this.
By essentially, because we can handle the errors quite gracefully,
error handling and go is just like any other value.
There's no exceptions that are thrown.
We can just pass them around, store them, retrieve them later.
We can also use context creatively.
You don't only have to use context and just pass it down to your calls,
but you can actually use them creatively.
The later cancellation of that work for you case, you can branch them out, you can join them together.
All of that really helps you structure your application.
Especially when building servers, the net HDP packages your friend.
There's a lot of in gems in there, as well as about controlling how your response is done.
And, for example, handling their request body.
So, really like if you're running into issues with it,
just like look for the documentation will probably find something in there that helps you.
And lastly, if you're faced with a problem that you say,
okay, my application, my benefit from original uploads,
you don't actually have to implement this again.
We have implementations in many different languages that are provided by us or the community.
So, just head over to Test.io if you're really interested in this.
But yeah, that's all I have.
Thank you very much for your attention.
Thank you.
Finally, all my uploads are saved.
I have time for one question if you're taking it.
Sure.
Anyone who troubles with their uploads?
I have a question.
I have one question, I'll come over with a microphone.
Keep raising your answer.
So, if you have partial uploads, for example,
and let's say it gets interrupted, but it never finishes,
how do a determine currently for some clean-up job,
you know, when to sweep them and how would you handle it?
Yeah.
So, when you clean up uploads that are not finished,
kind of depends on your application, right?
There's some cases where you want to give the client just a few seconds to retry,
but there's also cases where you want to give it, like,
for example, a day to complete, right?
So, it depends on that case.
What is important is that if you want to clean up,
you want to make sure that the upload resource
does not have concurrent access.
So, what we usually use is we have a distributed lease or lock mechanism
for that to make sure that the cleanup job doesn't interrupt
with any running requests.
We use a lot of context for making sure that all of the signals
get propagated around, but it's basically similar
that it feeds back into the request context as well.
Like, we close the context and the request stream
when we realize that we want to terminate the upload
because we're deleting the resource.
Thank you.
Thank you so much, applause.