Is Qwik + RxJS actually possible?

YouTube video for this article

I was very sad to hear that RxJS could not be serialized in Qwik. I invested a lot in learning RxJS, and I came to love writing reactive code.

But then I dived into Qwik to see what kind of reactivity was possible. Maybe something like React hooks? React doesn't need RxJS because of hooks, so maybe Qwik would be the same. This is the case, it turns out. Here's a YouTube video of me learning Qwik and coming to this conclusion:

But JLarky saw my video and wanted to help me try harder to get RxJS working in Qwik. We came to a better understanding of how it could work in this video, and in this article I want to summarize and expand on what we learned.

Serialization in Qwik

The first thing to understand is that everything that is code-split can only access either other exports, or serialized state from a component. So if you have this component:

export default component$(() => {
  // Code 1

  useWatch$(({ track }) => {
    // Code 2
  });

  return (
    <button onClick$={() => {
      // Code 3
    }}>Set to 1</button>
  );
});

The Qwik compiler will split this into 3 chunks containing Code 1, Code 2 and Code 3. Code 2 and 3 can only use stuff that is serializable from Code 1, and Code 1 can only use stuff from outside the component that is exported. This means these will throw errors:

const x = 1; // Need to export

export default component$(() => {
  return <div>{x}</div>; // Error
});

export default component$(() => {
  // Need to move out of component, export:
  const getX = () => 5; // Can't be serialized

  useWatch$(({ track }) => {
    const x = getX(); // Error
  });

return (
    <button onClick$={() => {
      const x = getX(); // Error
    }}>Set to 1</button>
  );
});

But these will work:

export default component$(() => {
  const getX = () => 5;
  return <div>{getX()}</div>; // No serialization needed, no error
});

export const getX = () => 5; // Exported

export default component$(() => {
  useWatch$(({ track }) => {
    const x = getX(); // No error
  });

return (
    <button onClick$={() => {
      const x = getX(); // No error
    }}>Set to 1</button>
  );
});

export default component$(() => {
  const x = 5; // Can be serialized

  useWatch$(({ track }) => {
    console.log('x', x); // No error
  });

return (
    <button onClick$={() => {
      console.log('x', x); // No error
    }}>Set to 1</button>
  );
});

Async Serialization

Ideally, a Qwik app can be paused at any time, transferred somewhere else, and then resumed again in the exact same state it was in before it was paused.

But Qwik can only pause behavior that is serializable, and this leaves out asynchronous stuff. We will need to find a way to represent the async behavior as serialized state in order to preserve it across a pause and resume. If we can figure this out for a simple setTimeout, maybe we can figure it out for some RxJS too.

Let's say we have a typeahead with a ridiculously long debounce of 3 seconds. If Qwik were to pause the application after 1.5 seconds, the debounce's internal timeout's progress would be lost, since it is not serialized as state.

If we are setting a timeout in the onKeyUp$ handler, I'm not aware of any way to recover the information about the timeout. This means the user could type something in the search, and if the app were paused and resumed immediately afterwards, the search results would never be fetched.

Alternatively, we can set a timeout inside either a useWatch$ or useClientEffect$ hook. When the user triggers an event, we can put the event data in a Qwik store, which will become serialized so the event data can never be lost. Then we can handle any async behavior downstream in a hook. The hooks also let us define a cleanup function that will be called when the app is being serialized, which we might be able to use to resume async behavior instead of resetting it.

But even if we find a way to access timeout data mid-timeout, Qwik doesn't give us any way to save it. If we try to set state during cleanup, it will be missed during serialization, instead triggering a re-render. If this happens on the server, this will be logged:

QWIK WARN Can not rerender in server platform

So we can't serialize a simple timeout.

At this point, trying to serialize an entire RxJS pipeline seems impossible.

Promises

Promises are not normally serializable, but believe it or not, Qwik has a way to serialize them. How does this work, and can we use it to help with RxJS?

Qwik is similar to React in a lot of ways, but it has a very important difference: Qwik renders asynchronously. This means you can throw a promise anywhere in a component, and the component will eventually render with the resolved promise. So this:

export default component$(() => {
  const message = new Promise(resolve => {
    setTimeout(() => resolve('Hi'), 1000);
  });

  return (
    <div>{message}</div>
  );
}); 

will take 1 second to render on the server, and then when it gets sent to the client you will see the resolved value in the rendered HTML: <div>Hi</div>

Even if we don't use the promise in the HTML, but in a useWatch$ for example, Qwik waits for it to resolve before serializing it. So you might say Qwik doesn't serialize any promise, but resolved promises specifically. This may change when Qwik implements out-of-order streaming, but I don't really understand this. If someone knows, please comment.

So instead of serializing ongoing asynchronous behavior, should we use promises to have Qwik wait for the behavior to complete and then serialize the result?

If this was good enough for RxJS, we could just use promises instead of RxJS in the first place. Some streams are never meant to complete, so converting them to a promise with lastValueFrom wouldn't preserve the desired behavior. The app would just never render.

It's great this async rendering capability exists in Qwik, but we still don't have a way to serialize RxJS streams.

Unserialized RxJS

What if we just ignore serialization for async behavior? If our app only renders once on the server, and never needs to be paused after it gets to the client, or if it's okay to completely drop asynchronous tasks, we will never need to serialize anything asynchronous. In practice, this should be good enough for most apps.

Let's implement our typeahead in RxJS without bothering with serialization and see what we come up with.

We would still like the RxJS code to be lazy-loaded, since the typeahead doesn't need to do anything until the user interacts. So we should avoid useClientEffect$, since it runs and loads the code immediately when the component becomes visible in the UI. It is only when the user interacts that we want to trigger the code to load, so we'll use a useWatch$.

Since components are constantly re-rendering in Qwik, like in React, we want to use RxJS the same way we would use it in React: Handle the asynchronous logic, set state in a single place, then let the app synchronously derive downstream state without any RxJS or subscriptions.

In React, this is the order of events we would want:

1. Component begins rendering
2. RxJS stream is subscribed to
3. User inputs
4. RxJS does its thing, sets state
5. Component rerenders, maintains subscription

In Qwik, this is the order we want:

1. Component renders on the server
2. App is serialized
3. HTML is transferred to the browser
4. User types, search is saved as state in a Qwik store
5. Qwik's reactivity loads and runs a useWatch$ that creates a BehaviorSubject to push values from the Qwik store to, and a series of RxJS chains lazy-load with a subscription getting called at the end
6. The value gets processed by the stream, and the output value gets put in the last Qwik store by the subscription
7. The component rerenders

This was complicated to implement, but here's a typeahead that uses some utilities I came up with:

export default component$(() => {
  const search = useSubject('');

  const results = useStream([] as Item[]);
  useWatch$((args) => {
    connect(
      [args, [search], results],
      () => rx(search).pipe(
        debounceTime(500),
        filter((search) => !!search.length),
        distinctUntilChanged(),
        switchMap((search) => fetchItems(search))
      )
    );
  });

  useSubscribe(results);

  return (
    <>
      <input
        style={{ border: "1px solid #08f" }}
        onKeyUp$={(e, t: HTMLInputElement) => next(search, t.value)}
      />
      {results.value.map((item) => (
        <div>{item.label}</div>
      ))}
    </>
  );
});

This achieves the exact behavior we want: No JavaScript is loaded until the user types, at which point the RxJS stream is loaded and passed the values being typed.

The RxJS stream has to be defined in the component inside a useWatch$, because the useWatch$ needs to manage the stream's subscription but could not access it if it were passed as a parameter of a utility function, since it would need the pipe to be serialized. But defining the pipe right in the useWatch$ doesn't require serialization. This makes using RxJS possible, but the drawback is we can't abstract the store/effect pair away into a a custom hook that we just pass the RxJS into, like we would be able to in React.

I defined my own next function for passing values into the store so I could reuse some logic that integrates the Qwik store with the BehaviorSubject.

How does it work?

I ended up supporting 4 special behaviors:

1. User input triggers RxJS to load
2. Observable (like a timer) is subscribed to independently, downstream RxJS lazy-loads only when it emits its first value
3. A subscription is created inside a useClientEffect$, causing the full RxJS chain to load as soon as the component is visible
4. An existing Qwik store pushes values into an observable, RxJS lazy-loads

In order to cover all of these scenarios, I had to do quite a bit of work.

For #1 I had to combine a Qwik store with a BehaviorSubject in a way that would allow downstream useWatch$s to load only when the first value was pushed.

#2 wasn't so bad. I just defined a convenient function that subscribes to an observable and sets it in a Qwik store/BehaviorSubject combination.

For #3, I had to define an activated property so an eager subscription could cause a bottom-up chain reaction of all observable dependencies at each level lazy-loading so that level could define its stream in terms of those dependencies. This was actually also required for #1 because a 2nd order stream could have multiple BehaviorSubjects as dependencies, so if the 1st one emits, in order to define the 2nd order stream we'd need to trigger the 2nd BehaviorSubject to get created.

#4 was pretty simple too. I just track the Qwik store property that needs to be converted into an RxJS stream inside a useWatch$ and call next on a useSubject store with the updated value.

I will publish my source code for these utilities when I write my next article. For now, take a look at the YouTube video for this article if you're interested in seeing a little more detail. (I haven't published the video yet. Subscribe to my YouTube channel and it should be up within a day or two :)

Eventually I'll try to get Redux Devtools involved too, which I'll implement in StateAdapt.

But should we?

I was not completely satisfied with the API I ended up with. Maybe we should just create an ecosystem of reactive custom hooks instead of using RxJS in Qwik. We should be able to do it in a more minimal way than this RxJS solution. It's possible to duplicate all the behavior of RxJS, just like in React.

I don't know the answer. But it's nice to know that at least RxJS is an option.