Author: John Au-Yeung

(The source code is at https://github.com/jauyeunggithub/bravado-quest)

Sometimes, we want to load large amounts of data in the background in a Nuxt app with web workers

In this article, we’ll look at how to load large amounts of data in the background in a Nuxt app with web workers

How to access route parameters in a page with Nuxt?

To run background tasks in a Nuxt app with web workers, we can add the Webpack’s worker-loader package into our Nuxt project.

Also, we’ve to make sure that web workers are only loaded on client side.

We’ll make a project that loads a large JSON file and searches it.

To start, we create the Nuxt project with:

npx create-nuxt-app quest

Then we run:

cd quest
npm run dev

to change to the project folder and run the project.

Next, we add Vuetify into the project with:

npm install @nuxtjs/vuetify -D

And we add the worker-loader package with:

npm i worker-loader@^1.1.1

To store data with IndexedDB, we install the Dexie package.

To install it, we run:

npm i dexie

Next, in nuxt.config.js, we change it to:

export default {
  // Global page headers: https://go.nuxtjs.dev/config-head
  head: {
    title: 'quest',
    htmlAttrs: {
      lang: 'en',
    },
    meta: [
      { charset: 'utf-8' },
      { name: 'viewport', content: 'width=device-width, initial-scale=1' },
      { hid: 'description', name: 'description', content: '' },
      { name: 'format-detection', content: 'telephone=no' },
    ],
    link: [{ rel: 'icon', type: 'image/x-icon', href: '/favicon.ico' }],
  },

  // Global CSS: https://go.nuxtjs.dev/config-css
  css: [],

  // Plugins to run before rendering page: https://go.nuxtjs.dev/config-plugins
  plugins: [
    { src: '~/plugins/inject-ww', ssr: false }
  ],

  // Auto import components: https://go.nuxtjs.dev/config-components
  components: true,

  // Modules for dev and build (recommended): https://go.nuxtjs.dev/config-modules
  buildModules: ['@nuxtjs/vuetify'],

  // Modules: https://go.nuxtjs.dev/config-modules
  modules: [],

  // Build Configuration: https://go.nuxtjs.dev/config-build
  mode: 'spa',
  build: {
    extend(config, { isClient }) {
      config.output.globalObject = 'this'

      if (isClient) {
        config.module.rules.push({
          test: /\.worker\.js$/,
          loader: 'worker-loader',
          exclude: /(node_modules)/,
        })
      }
    },
  },
  ssr: false,
}

We added the build.extend method to invoke the worker-loader when any file that ends with .worker.js is found.

isClient makes sure workers only load on client side apps.

config.output.globalObject = 'this' is needed so the hot reloading will work with the web workers loaded.

We also set ssr to false to disable server side rendering.

Also, we add { src: '~/plugins/inject-ww', ssr: false } to add the /plugins/inject-ww to load the worker loading plugin.

We add '@nuxtjs/vuetify' to load Vuetify in Nuxt.

In the plugins folder, we add:

import Worker from '~/assets/js/data.worker.js'

export default (_, inject) => {
  inject('worker', {
    createWorker () {
      return new Worker()
    }
  })
}

to load the worker from the /assets/js/data.worker.js.

To create worker, we create the data.worker.js file in the /assets/js/ and add:

import db from '@/db'

onmessage = async () => {
  if ((await db.avatars.toCollection().toArray()).length > 0) {
    postMessage({
      loaded: true,
    })
    return
  }
  const usersObj = await import(`@/data/users.json`)
  const users = Object.values(usersObj)
  const usersWithId = users.map((u, id) => ({ ...u, id }))
  await db.avatars.bulkPut(usersWithId)
  postMessage({
    loaded: true,
  })
}

@/data/users.json is a big JSON file saved in the project.

We import the users.json file from the data folder and store it in Indexed DB with Dexie.

We can load very large JSON files without crashing the browser since web workers run in a separate thread from the main browser thread.

db comes from db.js, which has:

import Dexie from 'dexie'

const db = new Dexie('db')
db.version(1).stores({
  avatars: '++id, address, avatar, city, email, name, title',
})

export default db

We create the db database with the avatars collection with the given keys.

We call bulkPut to write the retrieved data to the collection.

Now we cache the data in Indexed DB so they don’t have to load every time we load the app.

We make sure we try to load the cached data first with:

if ((await db.avatars.toCollection().toArray()).length > 0) {
  postMessage({
    loaded: true,
  })
  return
}

And we call postMessage to communicate that loading is done.

The message will be picked by by the message event handler when we invoke this worker.

In the pages, folder, we add _keyword.vue, to add an input and the search results component:

<template>
  <v-app>
    <v-card width="600px" elevation="0" class="mx-auto">
      <v-card-text :elevation="0">
        <v-text-field
          v-model="keyword"
          hide-details
          :prepend-inner-icon="mdiMagnify"
          full-width
          solo
          dense
          background-color="#FAFAFA"
        />

        <SearchResults :keyword="keyword" />
      </v-card-text>
    </v-card>
  </v-app>
</template>

<script >
import { mdiMagnify } from '@mdi/js'
import SearchResults from '@/components/SearchResults'

export default {
  components: {
    SearchResults,
  },
  data() {4
    return {
      keyword: '',
      mdiMagnify,
    }
  },
  beforeMount() {
    this.keyword = this.$route.params.keyword
  },
}
</script>

<style>
html {
  overflow: hidden;
}
</style>

The name of the page starts with an underscore means that we can get the URL parameter with the file name as the property name with the URL parameter value, so we can access the URL parameter value with the this.$route.params.keyword property.

Then in components/SearchResults.vue that we created, we add:

<template>
  <div>
    <div v-if="loading" class="my-3">
      <v-card>
        <v-card-text> Loading... </v-card-text>
      </v-card>
    </div>
    <div v-else id="scrollable" class="my-3">
      <v-card
        v-for="r of filteredSearchResultsWithHighlight"
        :key="r.id"
        class="my-3"
        :style="{
          border: highlightStatus[r.id] ? '2px solid lightblue' : undefined,
        }"
        @click="
          highlightStatus = {}
          $set(highlightStatus, r.id, !highlightStatus[r.id])
        "
      >
        <v-card-text class="d-flex pa-0 ma-0">
          <img :src="r.avatar" class="avatar" />
          <div
            class="flex-grow-1 pa-3 d-flex flex-column justify-space-between right-pane"
          >
            <div class="d-flex justify-space-between">
              <div>
                <h2 v-html="r.name"></h2>
                <p class="py-0 my-0">
                  <b v-html="r.title"></b>
                </p>
                <p class="py-0 my-0">
                  <span v-html="r.address"></span>,
                  <span v-html="r.city"></span>
                </p>
              </div>
              <div v-html="r.email"></div>
            </div>

            <div>
              <v-btn text color="#00897B">Mark as Suitable</v-btn>
            </div>
          </div>
        </v-card-text>
      </v-card>
    </div>
  </div>
</template>

<script>
import db from '@/db'

export default {
  props: {
    keyword: {
      type: String,
      default: '',
    },
  },
  data() {
    return {
      highlightStatus: {},
      filteredSearchResults: [],
      loading: false,
    }
  },
  computed: {
    filteredSearchResultsWithHighlight() {
      const { keyword } = this
      if (!Array.isArray(this.filteredSearchResults)) {
        return []
      }
      const highlighted = this.filteredSearchResults?.map((u) => {
        const highlightedEntries = Object.entries(u)?.map(([key, val]) => {
          if (key === 'avatar' || key === 'id') {
            return [key, val]
          }
          const highlightedVal = val?.replace(
            new RegExp(keyword, 'gi'),
            (match) => `<mark>${match}</mark>`
          )
          return [key, highlightedVal]
        })
        return Object.fromEntries(highlightedEntries)
      })
      return highlighted
    },
  },
  watch: {
    keyword: {
      immediate: true,
      handler() {
        this.search()
      },
    },
  },
  beforeMount() {
    this.loadData()
  },
  methods: {
    async search() {
      await this.$nextTick()
      const { keyword } = this
      if (keyword) {
        const filteredSearchResults = await db.avatars
          .filter((u) => {
            const { address, city, email, name, title } = u
            return (
              address?.toLowerCase()?.includes(keyword?.toLowerCase()) ||
              city?.toLowerCase()?.includes(keyword?.toLowerCase()) ||
              email?.toLowerCase()?.includes(keyword?.toLowerCase()) ||
              name?.toLowerCase()?.includes(keyword?.toLowerCase()) ||
              title?.toLowerCase()?.includes(keyword?.toLowerCase())
            )
          })
          .limit(10)
          .toArray()
        this.filteredSearchResults = filteredSearchResults
      } else {
        this.filteredSearchResults = await db.avatars
          ?.toCollection()
          ?.limit(10)
          .toArray()
      }
    },
    loadData() {
      this.loading = true
      const worker = this.$worker.createWorker()
      worker.onmessage = () => {
        this.search()
        this.loading = false
      }
      worker.postMessage('load')
    },
  },
}
</script>

<style>
.avatar {
  background: #bdbdbd;
  width: 150px;
}

.right-pane {
  background: #fafafa;
}

mark {
  background: yellow;
}

#scrollable {
  height: calc(100vh - 100px);
  overflow-y: auto;
}
</style>

In the loadData method. we call this.$worker.createWorker to create the worker that we injected.

this.$worker is available since the inject-ww.js script is run when the app is loaded.

We set worker.onmessage to a function so that we recent messages from the worker when postMessage in the onmessage function of the worker is called.

Once we received a message from the web workwer, we call the this.search method to do the filtering according to the keyword value.

And we call worker.postMessage with anything so that the worker’s onmessage function in the worker will start running.

Now when we type in something into the text box, we see results displayed in the cards.

We should be able to load large amounts of data in the web worker without hanging the browser since it’s run in the background.

Conclusion

To run background tasks in a Nuxt app with web workers, we can add the Webpack’s worker-loader package into our Nuxt project.

Also, we’ve to make sure that web workers are only loaded on client side.

We’ll make a project that loads a large JSON file and searches it.