Fast JS Search on Gatsby: Roll Your Own Site Search Fast JS Search on Gatsby: Roll Your Own Site Search
This Gatsby post was written for Gatsby 3 and is no longer maintained. If you would like to try out SvelteKit or Astro, check out the maintained posts which are based on those. Astro lets you continue to use React but also with partial hydration which can be used to provide an enhanced user experience. SvelteKit offers server side rendering as well as static site generation. See the post on how performance improved switching from Gatsby to Astro for more background.
In this post we look at how to add fast JS search on Gatsby. We are talking about site search here. That is allowing users to search for content on your site from within your site. Of course you could use one of the popular search services to provide the search functionality. However if you are the kind of developer who likes to get their hands dirty and roll their own functionality, this post is for you. We will even see how you can keep your site static while providing site search using JS Search.
JS Search was developed by Brian Vaughan from Facebook's core React team . Based on Lunr, it is so lightweight that we can build a search index just in time. Despite being lightweight, JS Search is highly configurable. We take advantage of the configurability to add stemming so that we return similar results. As an example, if the user searches for “like”, we return results which match likes, liked, likely and liking. If all of this sounds like your cup of tea, let's get going.
We will make search available to the user via the main home page on the
example.com/ route. Search will be triggered when an
s search parameter is included
in the URL. So if the use goes to
regular home page is loaded. However, if instead they go to
https://example.com/?s=cheese, we will render the search component and show search results for “cheese”. Of
course, the user will not need to know this to trigger a search — when the use clicks the
search button, we will direct them to the search page (by navigating to
https://example.com/?s= programatically). Let's look at how we will implement this.
- We will use the Climate Gatsby Starter, so we can hit the ground running and save ourselves pasting in boiler plate code. If you are adding search to an existing site, it should still be easy to follow along.
- Our first step will be to make the search documents available to the home page component, so it can build an index when needed. JS Search is so fast that there is no real advantage to building the search index ahead of time and serialising it. We will make the documents available using Gatsby's onCreatePage API to add the documents to the home page's context.
- Next we will create a Search component with placeholder content and add a search button to our nav bar.
- With all the preliminaries out of the way we can update our home page to render the Search component instead of its regular content whenever needed.
- Finally we can get into the JS Search part.
If you have followed one of these tutorials before, you know we like to add a little polish! How about some first class user experience (UX)? Wouldn't it be great if when the user opens up a search result the search term is highlighted in the text? On top, we will scroll the first occurrence of the search term into view.
Does that sound like a plan? Let's get to it then!
First of all let's clone the Gatsby Starter Climate repo. This is an MDX blog starter with some dummy content. Essentially we just need to add site search. To clone the repo and fire it up type these commands in the terminal:
We will need a few packages to add search functionality, let's add them all now:
Your new MDX blog with demo content should now be running at localhost:8000/ . The site is already fully functional and you would start customising it now with your own content if you weren't already doing this tutorial. Take a few moments to familiarise yourself with the project files and once you've seen what's what, let's carry on.
To tinker with the
onCreatePage API, we need to create a
gatsby-node.js file in the project's root folder:
What are we doing here? Before we get on to the
onCreatePage API we
need to make the post slug available, using the
. We will use it in
onCreatePage so that later, when a user clicks
a search result, they can be directed to the right post. Speaking of
onCreatePage, lets add this code to the end of
onCreatePage as the name suggests, runs for every page, during the
build process. We just need to add our search documents to the home page. We use a regular expression
20 only to run the code block for the home page. The following
21) gets all of the site's MDX blog posts.
One by one we will pull off data from the blog posts which we want users to be able to search for.
36, you can see
we are pullling data from the post frontmatter. The preceding code extracts the post body from its
original MDX format (which includes the frontmatter as well as the post body itself). We place
this data for each post into an array, along with an id which will we useful later. Finally in
lines 49–58, we replace the page object, adding a context. The context, like Gatsby's
GraphQL API makes data available to site pages. We will see shortly how we access this data in the
page's React component.
That's all the backend stuff complete. Next, let's add some UI.
Let's create a placeholder Search component. We will fill in content and logic later. Create a
Search.jsx file in the
Then we need to add a search button to the layout header, so let's edit
src/components/Layout.jsx. First we'll update imports:
Next we want to show a search button on every page, except when the
Search component is rendered. We will use a new prop,
hideSearch to help:
Then let's add the search button itself to the existing nav in the header:
Here we use Gatsby's
navigate function to navigate programatically
to the search page.
To finish off in this file, let's update the prop types:
This won't work yet because the
SearchIcon we import doesn't yet exist.
Let's create a search icon using Feather icons in the
OK, we just need to make the Search button look nice, before we move on:
If you click the search button, the URL changes, but nothing interesting happens. Let's change that in the following section.
Earlier we added a little data to our home page's context. Let's take a look at that data now.
Now in your browser go to localhost:8000/?s=anything
. You should see the regular home page content replaced. Instead you will see the Search
component with it's placeholder content. Below that, we are rendering the context data we made
available in the
onCreatePage API. Isn't that awesome? If you're happy
with how the context API works now, delete line
A couple of notes. In line
34 we are use the
to access the search parameters from the browser address bar. The check in line
33 just makes sure this code does not execute on the server while the page is building, as it could
break the build process (there is no browser address available during server build).
Before we move on let's update the prop types:
The final step in this post is to wire up the search component. Why don't we limber up for the home straight?
Let's define a search options file before we crack open the Search component:
These are options we will pass to JS Search, but what do they do?
- indexStrategy: Prefix match, the default, returns a match for “cheese” with “c”, “ch”, “che” etc. Other alternatives allow substring matches from any part of the word, not just the start (so the algorithm returns a match for “cheese” with “eese”). Finally you can request an exact match, so only “cheese” returns a match for “cheese”.
- searchSanitiser: whether we want to have case sensitive search or not. Lower case corresponds to case insentitive (“ChEeSe” matches “cheese”)
- indexBy: This tells JS Search which of the document data to index documents by.
- termFrequency: true switches on inverse document frequency which basically means very common words do not artificially push results higher up the results list.
- stopWords: The algorithm ignores stop words like and, that and the which are unlikely to be important for the search query when we set this option to true.
- stemWords: We mentioned this earlier. Going back to “cheese”, the algorithm would consider, “cheesy” a match for “cheese”.
With that out of the way, let's replace the Search component placeholder code:
Let's look at this a bit at a time.
39, we import
the options we just defined. Then in lines
45 we get the search parameter, this is similar to what we
did on the home page. We need to do this so we know what the user intends to search for. Later we will
set the URL based on the search term the user enters into the search box. The search parameter is the
single source of truth, when it come to the user's search term.
107 configure the search using the options we just set. We
useEffect API with an empty array as the second argument,
so that the search indices are only built once, when the component mounts in lines
110. When JS Search builds the search index, it generates
the search object and stores this in the component's state (line
104). Now whenever we have a search query, we just use the search object (which has the index
embedded) to get our search results.
What happens when the user types a search query in the search box? Let's look at that step, by
step. With each key press the handleChange method is called. We could add a throttle function here
but JS Search is pretty fast so there's probably no need, unless we have a large number of
documents. Anyway, the
handleChange method (lines
130), gets the search results from our search object,
sets the UI to display the search results and changes the browser URL to match the search query.
If there are some search results, we display a
for each result. This is the same component we use on the regular home page, just instead of feeding
it all posts in date order, we display search matches, most relevant first.
That was a huge chunk of code, and we only looked at the most important parts, so do let me know, if some aspects need a touch more clarification.
Try searching for “camera”. This is in every post so all three dummy posts appear in results. Now if we start over, this time searching for “fold”, you see the two posts where folding cameras are mentioned appear. Click a result and the browser opens it up for you. Next search for “Elon Musk” — no results! Looks like it's all working.
In this post we learned:
- about JS Search configuration options,
how we can pass data to a component's context using Gatsby's
- using the above to add search to your Gatsby site.
We are still missing the UX polish. No need to worry though, we will add the shine in this follow-up post on highlighting search results using Web APIs . You can see the full code for this JS Search on Gatsby on the Rodney Lab Git Hub repo . You can visit the live demo site at gatsby-site-search.rodneylab.com .
Keen to hear how you plan to use this! Is there something from this post you can leverage for a side project or even client project? I do hope so! Let me know if there is anything in the post that I can improve on, for anyone else creating this project. You can leave a comment below, @ me on twitter or try one of the other contact methods listed below.
If you have found this post useful, even though you can only afford even a tiny contribution, please consider supporting me through Buy me a Coffee .
Finally, feel free to share the post on your social media accounts for all your followers who will find it useful. As well as leaving a comment below, you can get in touch via @askRodney on Twitter and also askRodney on Telegram . Also, see further ways to get in touch with Rodney Lab . I post regularly on SvelteKit as well as Gatsby JS among other topics. Also subscribe to the newsletter to keep up-to-date with our latest projects.