MAIS AMOR POR FAVOR

SvelteKit S3 Compatible Storage: Presigned Uploads

SvelteKit S3 Compatible Storage

SvelteKit S3 Compatible Storage: Presigned Uploads

SHARE:

πŸ˜• Why S3 Compatible Storage?

In this post on SvelteKit compatible S3 storage, we will take a look at how you can add an upload feature to your Svelte app. We use presigned links, allowing you to share private files in a more controlled way. Rather that focus on a specific cloud storage provider's native API, we take an S3 compatible approach. Cloud storage providers like Backblaze, Supabase and Cloudflare R2 offer access via an API compatible with Amazon's S3 API. The advantage of using an S3 compatible API is flexibility. If you later decide to switch provider, you will be able to keep the bulk of your existing code.
SvelteKit S3 Compatible Storage: What we're building

We will build a single page app in SvelteKit which lets the visitor upload a file to your storage bucket. You might use this as a convenient way of uploading files for your projects to the cloud. Alternatively it can provide a a handy starting point for a more interactive app, letting users upload their own content. That might be for a photo sharing app, your own micro-blogging service or for an app letting clients preview and provide feedback on your amazing work. I hope this is something you find interesting if it is let's get going.

βš™οΈ Getting Started

Let start by creating a new skeleton SvelteKit project. Type the following commands in the terminal:

pnpm init [email protected] sveltekit-graphql-github && cd $_
pnpm install

We will be using the official AWS SDK for some operations on our S3 compatible cloud storage. As well as the npm packages for the SDK we will need a few other packages including some fonts for self-hosting. Lets install all of these now:

pnpm i -D @aws-sdk/client-s3 @aws-sdk/s3-request-presigner @aws-sdk/util-create-request @aws-sdk/util-format-url @fontsource/libre-franklin @fontsource/rajdhani cuid dotenv

Initial Authentication

Although most of the code we look at here should work with any S3 compatible storage provider, the mechanism for initial authentication will be slightly different for each provider. Even taking that into account, it should still make sense to use the provider's S3 compatible API for all other operations to benefit from the flexibility this offers. We focus on Backblaze for initial authentication. Check your own provider's docs for their mechanism.

To get S3 compatible storage parameters from the Backblaze API you need to supply an Account ID and Account Auth token with read and write access to the bucket we want to use. Let's add these to a .env file together with the name of the bucket (if you already have one set up). Buckets offer a mechanism for organising objects (or files) in cloud storage. They play a role analogous to folders or directories on your computer's file system.

.env
plaintext
S3_COMPATIBLE_ACCOUNT_ID="your-account-id"
S3_COMPATIBLE_ACCOUNT_AUTH_TOKEN="your-auth-token"
S3_COMPATIBLE_BUCKET_NAME="your-bucket-name"

The last bit of setup before spinning up the dev server is to configure the dotenv environment variables package in svelte.config.js:

svelte.config.js
javascript
1import 'dotenv/config';
2
3/** @type {import('@sveltejs/kit').Config} */
4const config = {
5 kit: {
6 // hydrate the <div id="svelte"> element in src/app.html
7 target: '#svelte',
8 },
9};
10
11export default config;

Start the dev Server

Use this command to start the dev server:

pnpm run dev

By default it will run on TCP port 3000. If you already have something running there, see how you can change server ports in the article on getting started with SvelteKit.

πŸ”— Presigned URLs

We will generate presigned read and write URLS on the server side. Presigned URLs offer a way to limit access, granting temporary access. Links are valid for 15 Β minutes by default. Potential clients, app users and so on will be able to access just the files you want them to access. Also because you are using presigned URLs you can keep the access mode on your bucket set to private.

To upload a file we will use the write signed URL. We will also get a read signed URL. We can use that to download the file if we need to.

Let's create a SvelteKit server endpoint to listen for new presigned URL requests. Create a src/routes/api folder adding an presigned-urls.json.js file with the following content:

src/routes/api/presigned-urls.js
javascript
1import { GetObjectCommand, PutObjectCommand, S3 } from '@aws-sdk/client-s3';
2import { S3RequestPresigner } from '@aws-sdk/s3-request-presigner';
3import { createRequest } from '@aws-sdk/util-create-request';
4import { formatUrl } from '@aws-sdk/util-format-url';
5import cuid from 'cuid';
6
7const S3_COMPATIBLE_BUCKET = process.env['S3_COMPATIBLE_BUCKET_NAME'];
8const S3_COMPATIBLE_ACCOUNT_AUTH_TOKEN = process.env['S3_COMPATIBLE_ACCOUNT_AUTH_TOKEN'];
9const S3_COMPATIBLE_ACCOUNT_ID = process.env['S3_COMPATIBLE_ACCOUNT_ID'];
10
11async function authoriseAccount() {
12 try {
13 const authorisationToken = Buffer.from(
14 `${S3_COMPATIBLE_ACCOUNT_ID}:${S3_COMPATIBLE_ACCOUNT_AUTH_TOKEN}`,
15 'utf-8',
16 ).toString('base64');
17
18 const response = await fetch('https://api.backblazeb2.com/b2api/v2/b2_authorize_account', {
19 method: 'GET',
20 headers: {
21 Authorization: `Basic ${authorisationToken}`,
22 },
23 });
24 const data = await response.json();
25 const {
26 absoluteMinimumPartSize,
27 authorizationToken,
28 apiUrl,
29 downloadUrl,
30 recommendedPartSize,
31 s3ApiUrl,
32 } = data;
33 return {
34 successful: true,
35 absoluteMinimumPartSize,
36 authorizationToken,
37 apiUrl,
38 downloadUrl,
39 recommendedPartSize,
40 s3ApiUrl,
41 };
42 } catch (error) {
43 let message;
44 if (error.response) {
45 message = `Storage server responded with non 2xx code: ${error.response.data}`;
46 } else if (error.request) {
47 message = `No storage response received: ${error.request}`;
48 } else {
49 message = `Error setting up storage response: ${error.message}`;
50 }
51 return { successful: false, message };
52 }
53}

This code works for Backblaze's API but will be slightly different if you use another provider. The rest of the code we look at should work with any S3 compatible storage provider.

In lines 7–9 we pull the credentials we stored, earlier, in the .env file. Moving on, in lines 13–16 we see how you can generate a Basic Auth header in JavaScript. Finally, the Backblaze response returns a recommended and minimum part size. These are useful when uploading large files. Typically you will want to split large files into smaller chunks. These numbers give you some guidelines on how big each of the chunks should be. We look at presigned multipart uploads in another article. Most important though is the s3ApiUrl which we will need to create a JavaScript S3 client.

Creating Presigned Links with S3 SDK

Next we use that S3 API URL to get the S3 region and then use that to get the presigned URLs from the SDK. Add this code to the bottom of the storage.js file:

src/routes/api/presigned-urls.config.js
javascript
55function getRegion(s3ApiUrl) {
56 return s3ApiUrl.split('.')[1];
57}
58
59function getS3Client({ s3ApiUrl }) {
60 const credentials = {
61 accessKeyId: S3_COMPATIBLE_ACCOUNT_ID,
62 secretAccessKey: S3_COMPATIBLE_ACCOUNT_AUTH_TOKEN,
63 sessionToken: `session-${cuid()}`,
64 };
65
66 const S3Client = new S3({
67 endpoint: s3ApiUrl,
68 region: getRegion(s3ApiUrl),
69 credentials,
70 });
71 return S3Client;
72}
73
74async function generatePresignedUrls({ key, s3ApiUrl }) {
75 const Bucket = S3_COMPATIBLE_BUCKET;
76 const Key = key;
77 const client = getS3Client({ s3ApiUrl });
78
79 const signer = new S3RequestPresigner({ ...client.config });
80 const readRequest = await createRequest(client, new GetObjectCommand({ Key, Bucket }));
81 const readSignedUrl = formatUrl(await signer.presign(readRequest));
82 const writeRequest = await createRequest(client, new PutObjectCommand({ Key, Bucket }));
83 const writeSignedUrl = formatUrl(await signer.presign(writeRequest));
84 return { readSignedUrl, writeSignedUrl };
85}
86
87export async function presignedUrls(key) {
88 try {
89 const { s3ApiUrl } = await authoriseAccount();
90 const { readSignedUrl, writeSignedUrl } = await generatePresignedUrls({ key, s3ApiUrl });
91 return { readSignedUrl, writeSignedUrl };
92 } catch (error) {
93 console.error(`Error generating presigned urls: ${error}`);
94 }
95}

In line 63 we use the cuid package to help us generate a unique session id. That's the server side setup. Next let's look at the client.

πŸ—³οΈ Poll

Which is your preferred cloud storage provider?
Voting reveals latest results.

πŸ§‘πŸ½ Client Home Page JavaScript

We'll split the code into a couple of stages. First let's add our script block with the code for interfacing with the endpoint that we just created and also the cloud provider. We get presigned URLs from the endpoint then, upload directly to the cloud provider from the client. Since all we need for upload is the presigned URL, there is no need to use a server endpoint. This helps us keep the code simpler.

Replace the content of src/routes/index.svelte with the following:

src/routes/index.svelte
svelte
1<script>
2 import '@fontsource/rajdhani';
3 import '@fontsource/libre-franklin';
4
5 const H_ELLIPSIS_ENTITY = '\u2026'; // ...
6 const LEFT_DOUBLE_QUOTE_ENTITY = '\u201c'; // "
7 const RIGHT_DOUBLE_QUOTE_ENTITY = '\u201d'; // "
8
9 let isSubmitting = false;
10 let uploadComplete = false;
11 let files = [];
12 let errors = { files: null };
13 let downdloadUrl = '';
14 $: filename = files.length > 0 ? files[0].name : '';
15
16 function resetForm() {
17 files = [];
18 errors = { files: null };
19 }
20
21 const handleChange = (event) => {
22 errors = { files: null, type };
23 files = event.target.files;
24 };
25
26 const handleSubmit = async () => {
27 try {
28 if (files.length === 0) {
29 errors.files = 'Select a file to upload first';
30 return;
31 }
32
33 isSubmitting = true;
34 const { name: key } = files[0];
35
36 // get signed upload URL
37 const response = await fetch('/api/presigned-urls.json', {
38 method: 'POST',
39 credentials: 'omit',
40 headers: {
41 'Content-Type': 'application/json',
42 },
43 body: JSON.stringify({ key }),
44 });
45 const json = await response.json();
46 const { readSignedUrl, writeSignedUrl } = json;
47 downdloadUrl = readSignedUrl;
48
49 // Upload file
50 const reader = new FileReader();
51 reader.onloadend = async () => {
52 await fetch(writeSignedUrl, {
53 method: 'PUT',
54 body: reader.result,
55 headers: {
56 'Content-Type': type,
57 },
58 });
59 uploadComplete = true;
60 isSubmitting = false;
61 };
62 reader.readAsArrayBuffer(files[0]);
63 } catch (error) {
64 console.log(`Error in handleSubmit on / route: ${error}`);
65 }
66 };
67</script>

The first part is mostly about setting up the user interface state. There is nothing unique to this app there, so let's focus on the handleSubmit function. There are two parts. The first in which we get a signed URL from the endpoint we just created and the second where we use the FileReader API to upload the file to the cloud.

FileReader API

The FileReader API lets us read in a file given the local path and output a binary string, DataURL or an array buffer. You would use a DataURL if you wanted to Base64 encode an image (for example). You could then set the src of an <img> element to a generated Base64 data uri string or upload the image to a Cloudflare worker for processing. For our use case, uploading files to cloud storage, instead we go for the readAsArrayBuffer option.

The API is asynchronous so we can just tell it what we want to do once the file is uploaded and carry on living our life in the meantime! We create an instance of the API in line 50. Using onloadend we specifiy that we want to use fetch to upload our file to the cloud, once it is loaded into an array buffer (from the local file system). In line 62 (after the onreadend block), we specify what we want to read. The file actually comes from a file input, which we will add in a moment.

Fetch Request

The fetch request is inside the onloadend block. We make a PUT request, including the file type in a header. The body of the request is the result of the file read from the FileReader API. Because we are making a PUT request, from the browser, and also because the content type may not be text/plain, we will need some CORS configuration. We'll look at that before we finish.

How do we get the file name and type? When the user selects a file, from the file input we just mentioned, the handleChange code in lines 21–24 runs. This gets the file, by updating the files variable, but does not read the file in (that happens in our FileReader API code). Next, when the user clicks the Upload button which triggers the handleSubmit function call, we get the name and file content type in line 34.

πŸ–₯ Client Home Page Markup

Next we'll add the markup, including the file browse input which lets the user select a file to upload. After that we'll add some optional styling, look at CORS rules and finally test.

Paste this code at the bottom of the index.svelte file:

src/routes/index.svelte
svelte
69<svelte:head>
70 <title>SvelteKit S3 Compatible Storage</title>
71 <html lang="en-GB" />
72 <meta
73 name="description"
74 content="Upload a file to third party storage using an S3 compatible API in SvelteKit."
75 />
76</svelte:head>
77
78<main class="container">
79 <h1>SvelteKit S3 Compatible Storage</h1>
80 {#if uploadComplete}
81 <section class="upload-complete">
82 <h2 class="heading">Upload complete</h2>
83 <p class="filename">
84 Download link: <a aria-label={`Download ${filename}`} href={downdloadUrl}>{filename}</a>
85 </p>
86 <div class="button-container">
87 <button
88 class="another-upload-button"
89 on:click={() => {
90 uploadComplete = false;
91 resetForm();
92 }}>Upload another file</button
93 >
94 </div>
95 </section>
96 {:else}
97 <section class="upload">
98 <form on:submit|preventDefault={handleSubmit}>
99 <h2 class="heading">Upload a file{H_ELLIPSIS_ENTITY}</h2>
100 {#if filename !== ''}
101 <p class="filename">{filename}</p>
102 <p class="filename">
103 Click {LEFT_DOUBLE_QUOTE_ENTITY}Upload{RIGHT_DOUBLE_QUOTE_ENTITY} to start upload.
104 </p>
105 {/if}
106 {#if errors.files}
107 <div class="error-text-container">
108 <small id="files-error" class="error-text">{errors.files}</small>
109 </div>
110 {/if}
111 {#if isSubmitting}
112 <small id="files-error">Uploading{H_ELLIPSIS_ENTITY}</small>
113 {/if}
114 <div class="file-input-container">
115 <label class="file-input-label" for="file"
116 ><span class="screen-reader-text">Find a file to upload</span></label
117 >
118 <input
119 id="file"
120 aria-invalid={errors.files != null}
121 aria-describedby={errors.files != null ? 'files-error' : null}
122 type="file"
123 multiple
124 formenctype="multipart/form-data"
125 accept="image/*"
126 title="File"
127 on:change={handleChange}
128 />
129 <div class="button-container">
130 <button type="submit" disabled={isSubmitting}>Upload</button>
131 </div>
132 </div>
133 </form>
134 </section>
135 {/if}
136</main>

You can see the file input code in lines 118–128. We have set the input to allow the user to select multiple files (multiple attribute in line 123). For simplicity the logic we added previously only uploads the first file, though you can tweak it if you need multiple uploads from your application. In line 125 we set the input to accept only image files with accept="image/*". This can be helpful for user experience, as typically in the file select user interface, just image files will be highlighted. You can change this to accept just a certain image format or different file types, like PDF, or video formats β€” whatever your application needs. See more on file type specifier in the MDN docs .

SvelteKit S3 Compatible Storage: screen capture shows a custom file styled input, with a button labelled Browse.
SvelteKit S3 Compatible Storage: File Input

Finally before we check out CORS, here's some optional styling. This can be nice to add as the default HTML file input does not look a little brutalistic!

src/routes/index.svelte β€” click to expand code.
src/routes/index.svelte
svelte
138<style>
139 :global(html) {
140 background-image: linear-gradient(
141 to top right,
142 var(--colour-theme-lighten-20),
143 var(--colour-theme)
144 );
145 color: var(--colour-light);
146
147 font-family: Libre Franklin;
148 }
149
150 :global(:root) {
151 --colour-theme: #3185fc; /* azure */
152 --colour-theme-lighten-20: #4599ff;
153 --colour-light: #fafaff; /* ghost white */
154 --colour-light-opacity-85: #fafaffd9;
155 --colour-dark: #403f4c; /* dark liver */
156 --colour-feature: #f9dc5c; /* naples yellow */
157 --colour-alternative: #e84855; /* red crayola */
158 --font-weight-medium: 500;
159 }
160
161 .screen-reader-text {
162 border: 0;
163 clip: rect(1px, 1px, 1px, 1px);
164 clip-path: inset(50%);
165 height: 1px;
166 margin: -1px;
167 width: 1px;
168 overflow: hidden;
169 position: absolute !important;
170 word-wrap: normal !important;
171 }
172 .error-text-container {
173 margin: 2rem 0 0.5rem;
174 }
175 .error-text {
176 color: var(--colour-feature);
177 background-color: var(--colour-dark);
178 padding: 0.5rem 1.25rem;
179 border-radius: 1.5rem;
180 border: solid 0.0625rem var(--colour-feature);
181 }
182
183 .container {
184 margin: 1.5rem;
185 min-height: 100vh;
186 }
187
188 .container h1 {
189 font-family: Rajdhani;
190 font-size: 1.953rem;
191 }
192
193 .upload,
194 .upload-complete {
195 margin: 4rem 1rem;
196 padding: 1.5rem;
197 border: solid 0.125rem var(--colour-light);
198 border-radius: 0.5rem;
199 }
200
201 .button-container {
202 display: flex;
203 }
204
205 :is(.upload, .upload-complete) .heading {
206 font-family: Rajdhani;
207 font-size: 1.563rem;
208 margin-top: 0;
209 }
210
211 .upload-complete {
212 background-color: var(--colour-feature);
213 color: var(--colour-dark);
214 border-color: var(--colour-dark);
215 }
216 .filename {
217 margin-left: 1rem;
218 }
219
220 .filename a {
221 color: var(--colour-dark);
222 text-underline-offset: 0.125rem;
223 }
224
225 .file-input-container {
226 display: flex;
227 align-items: center;
228 justify-content: flex-end;
229 padding: 1.5rem 0 0.5rem;
230 }
231
232 .file-input-label::before {
233 content: 'Browse\2026';
234 margin-left: auto;
235 }
236
237 .file-input-label::before,
238 button {
239 font-family: Libre Franklin;
240 background: var(--colour-theme);
241 cursor: pointer;
242 color: var(--colour-light);
243 border: solid 0.0625rem var(--colour-light);
244 border-radius: 1.5rem;
245 margin-left: 1rem;
246 padding: 0.5rem 1.75rem;
247 font-size: 1.25rem;
248 font-weight: var(--font-weight-medium);
249 }
250
251 @media (prefers-reduced-motion: no-preference) {
252 .file-input-label::before,
253 button {
254 transition: background-color 250ms, color 250ms;
255 }
256 }
257 @media (prefers-reduced-motion: no-preference) {
258 .file-input-label::before,
259 button {
260 transition: background-color 2000ms, color 2000ms;
261 }
262 }
263
264 button:hover,
265 .file-input-label:hover:before,
266 button:focus,
267 .file-input-label:focus:before {
268 background-color: var(--colour-light-opacity-85);
269 color: var(--colour-dark);
270 }
271
272 .another-upload-button {
273 margin-left: auto;
274 }
275
276 .upload-complete button:hover,
277 .upload-complete button:focus {
278 border-color: var(--colour-dark);
279 }
280
281 input[type='file'] {
282 visibility: hidden;
283 width: 1px;
284 }
285
286 @media (min-width: 768px) {
287 .container {
288 margin: 3rem 1.5rem;
289 }
290
291 .upload,
292 .upload-complete {
293 margin: 4rem 10rem;
294 }
295 }
296</style>

β›” Cross-Origin Resource Sharing (CORS)

CORS rules are a browser security feature which limit what can be sent to a different origin. By origin we mean sending data to example-b.com when you are on the example-a.com site. If the request to a cross origin does not meet some basic criteria (GET request or POST with text/plain content type, for example) the browser will perform some extra checks. We send a PUT request from our code so the browser will send a so-called preflight request ahead of the actual request. This just checks with the site we are sending the data to what it is expecting us to send, or rather what it will accept.

To avoid CORS issues, we can set CORS rules with our storage provider. It is possible to set them on your bucket when you create it. Check with your provider on the mechanism for this. With Backblaze you can set CORS rules using the b2 command line utility in JSON format. Here is an example file:

backblaze-bucket-cors-rules.json
json
1[
2 {
3 "corsRuleName": "development",
4 "allowedOrigins": ["https://test.localhost.com:3000"],
5 "allowedHeaders": ["content-type", "range"],
6 "allowedOperations": ["s3_put"],
7 "exposeHeaders": ["x-amz-version-id"],
8 "maxAgeSeconds": 300
9 },
10 {
11 "corsRuleName": "production",
12 "allowedOrigins": ["https://example.com"],
13 "allowedHeaders": ["content-type", "range"],
14 "allowedOperations": ["s3_put"],
15 "exposeHeaders": ["x-amz-version-id"],
16 "maxAgeSeconds": 3600
17 }
18]

We can set separate rules to let our dev and production requests work. In the allowed origin for dev, we set a dummy hostname instead of localhost and on top we run in HTTPS mode. You may be able to have everything working without this setup, but try it if you have issues. Add this CORS configuration to Backblaze with the CLI utility installed by running:

b2 update-bucket --corsRules "$(cat backblaze-bucket-cors-rules.json)" your-bucket-name allPrivate

You can see more on Backblaze CORS rules in their documentation .

Secure dev Server

To run the SvelteKit dev server in https mode, update your package.json dev script to include the --https flag:

package.json
json
1{
2 "name": "sveltekit-s3-compatible-storage",
3 "version": "0.0.1",
4 "scripts": {
5 "dev": "svelte-kit dev --port 3000 --https",

Then restart the dev server with the usual pnpm run dev command. Learn more about this in the video on running a secure SvelteKit dev server.

To set a local hostname, on MacOS add a line to private/etc/hosts:

/private/etc/hosts
plaintext
1127.0.0.1 test.localhost.com

Then, instead of accessing the site via http://localhost:3030, in your browser use https://test.localhost.com:3030. This worked for me on macOS. The same will work on typical Linux and Unix systems, though the file you change will be /etc/hosts. If you are using DNSCryprt Proxy or Unbound, you can make a similar change in the relevant config files. If you use windows and know how to do this, please drop a comment below to help out other windows users.

πŸ’― SvelteKit S3 Compatible Storage: Test

Try uploading a file using the new app. Also make sure the download link works.

SvelteKit S3 Compatible Storage: screen capture shows text stating upload is complete and includes a like to download the file. There is also a button for uploading another file.
SvelteKit S3 Compatible Storage: Test

πŸ™ŒπŸ½ SvelteKit S3 Compatible Storage: What we Learned

In this post we learned:

  • why you would use the S3 compatible API for cloud storage instead of your storage provider's native API,

  • how to use the AWS SDK to generate a presigned upload URL,

  • a way to structure a file upload feature in a SvelteKit app.

I do hope there is at least one thing in this article which you can use in your work or a side project. As an extension you might want to pull a bucket list and display all files in the folder. You could even add options to delete files. On top, you could also calculate a hash of the file before upload and compare that to the hash generated by your storage provider. This avails a method to verify file integrity. There's a world of different apps you can add an upload feature to; knock yourself out!

You can see the full code for this SvelteKit S3 compatible storage project on the Rodney Lab Git Hub repo .

🏁 SvelteKit S3 Compatible Storage: Summary

What is S3 compatible storage?

AWS offer a cloud storage service called S3. The S3 API can be used to access storage on most other providers. Using this S3 API lets you store and retrieve your data from any of these providers in a standard way. Doing so gives you some flexibility, making it easier to change storage provider at a later date.

Why use a presigned URL?

A presigned URL offers a mechanism for granting temporary access to private files in your storage bucket on a per file basis. Using presigned URLs you can let your clients or site visitors download files from your private bucket.

How can you upload files to cloud storage in SvelteKit?

The easiest way is to use the HTML5 FileReader API. We saw how to do that from the browser on the client side of your SvelteKit app in this post. We also saw how to configure CORS for your bucket.

πŸ™πŸ½ SvelteKit S3 Compatible Storage: Feedback

Have you found the post useful? Would you prefer to see posts on another topic instead? Get in touch with ideas for new posts. Also if you like my writing style, get in touch if I can write some posts for your company site on a consultancy basis. Read on to find ways to get in touch, further below. If you want to support posts similar to this one and can spare a few dollars, euros or pounds, please consider supporting me through Buy me a Coffee.

Finally, feel free to share the post on your social media accounts for all your followers who will find it useful. As well as leaving a comment below, you can get in touch via @askRodney on Twitter and also askRodney on Telegram . Also, see further ways to get in touch with Rodney Lab. I post regularly on SvelteKit as well as other topics. Also subscribe to the newsletter to keep up-to-date with our latest projects.