Use Blitz.js superjson instead of native JSON.serialize/parse

[sergiodxa]

Note

This is not needed anymore now that Remix uses turbo-stream when Single Fetch feature is enabled, since that allows sending more values than plain JSON, like Date, BigInt, Map, etc.

What is the new or updated feature that you are suggesting?

Use superjson to serialize and parse data returned from loaders/actions.

Why should this feature be included?

Right now Remix uses JSON.stringify and JSON.parse, this means types like Date (see #634) are going to be converted to strings when serializing it but not converted back to Date on the revive step, and other types like BigInt are not even compatible with JSON.stringify, so if an ORM returns one the user needs to manually convert them to a string before returning the data/response inside the loader.

This superjson library created by the Blitz.js team have differents stringify and parse method that it's capable of serializing more types of data and reviving them back to the original class (see the table here https://github.com/blitz-js/superjson#parse), this way if a loader does this:

return json({ date: new Date() })

Then the useLoaderData will get this

let { date } = useLoaderData<{ date: Date }>()
let fullYear = date.getFullYear() // this will work

And other native objects like BigInt, RegExp, Set, Map and Error will also work as valid return values.

---

Additionally, superjson can be extended to serialize and revive other objects, it may be cool if Remix let you customize that somehow, but I think just the default config would be better than the native JSON methods.

[mikestopcontinues]

It's 8.7kB gzipped. I don't think it's worth including. It goes against the "web fundamentals" approach of Remix. Better to rally TC39 to update the JSON spec instead.

[kevlened]

If you're willing to fudge on the "web fundamentals" a bit, Rich Harris's devalue adds 0kB and it's faster. See a comparison here.

[thomasfr]

Out of curiosity, is there a proposal to extend the Spec? Could not find one here: https://github.com/tc39/proposals

[sergiodxa]

@mikestopcontinues it's tree-shakeable, so that's not the final size, since the browser only needs one method it should be smaller, but would need to be measured correctly

[mikestopcontinues]

For the record, consider me convinced!

[ghost]

Could this be supported by having serialize/parse hooks in the client/server entries?

[sergiodxa]

Could this be supported by having serialize/parse hooks in the client/server entries?

I don't think so, you can't change how Remix parse the data from the first request and for requests it does when navigating. This is something that Remix needs to do internally, or somehow expose hooks to do that which is not possible right now.

[ghost]

It would be similar to how tRPC does it: https://trpc.io/docs/data-transformers

[zachdtaylor]

Does somebody need to give the final word on this before someone can make a PR?

[sergiodxa]

@zachtylr21 I already made a PR #662

[zachdtaylor]

@sergiodxa in the future, is there a way to know which issues are already being worked on? I wonder if we should use the "assignees" section so people don't accidentally start working on the same thing. Or is it just assumed that if you create the issue, you create the PR as well?

[sergiodxa]

@zachtylr21 honestly, I don't know, I also don't know why GitHub didn't related the issue with the PR (I'm sure closes # and fix # worked for that), I think assignees makes sense but without someone on the team doing it no one can do it.

[zachdtaylor]

@sergiodxa Cool, maybe I'll just assume that if someone else created the issue, they're working on it, or I'll just ask them. Btw, I just looked at your PR, I think it's supposed to be "fixes" not "fix".

[sergiodxa]

@zachtylr21 just tried with fixes, it doesn't link them, I'm not sure what's happening with GitHub

[zachdtaylor]

@sergiodxa weird 🤷

[MichaelDeBoey]

I'm not sure what's happening with GitHub

@sergiodxa I think that's because you're not creating a PR against the default branch (which is main)

[sergiodxa]

I'm not sure what's happening with GitHub

@sergiodxa I think that's because you're not creating a PR against the default branch (which is main)

Ahhh that could be the reason, I think the team should set the default branch as dev so people open PRs automatically against that branch instead of main

[arthurgeek]

What is the proposed way to deal with this issue?

Right now, this is what I'm doing, but I'm sure there are better ways (I'm new to TypeScript) :

export type TransactionJson = Omit<Transaction, 'createdAt'> & {
  createdAt: string
}

export type LoaderResponse = {
  transactions: TransactionJson[]
}

export const loader: LoaderFunction = async ({ params }) => {
  invariant(params.accountId, 'expected params.accountId')

  const transactions = await getTransactions(params.accountId)

  return json({ transactions })
}

export default function Transactions() {
  const { transactions } = useLoaderData<LoaderResponse>()

  [...]
}

Good thing I need an string representation of that Date inside Transactions, otherwise I would need to map that Transaction[] to parse the string as a Date.

[mikestopcontinues]

@arthurgeek Separate from the superjson idea, I've always found it better to store timestamps and convert them to Date only when necessary. This was partly for JSON, but mostly just because they cause less headaches. That said, nowadays date-fns has made it so I almost never find myself reaching for Date.

[mwarnerdotme]

As someone who uses Maps and Sets in their (backend) code, I would appreciate support for serializing them.

[donferi]

Would love this, currently all my prisma date types are incorrect and it's incredibly frustrating.

[lucioladen]

+1
Is kind of annoying this native JSON serialization/parsing lack of support for very quite common types like bigint or dates, specially inuse cases dealing with ORM models (as you might expect, a lot of use cases).
@sergiodxa did a good job with a lighter-weight json() in his remix-utils package, but would be nice for this to be tackled by remix.

[n1ghtmare]

Constantly parsing dates is really annoying, this feature would be amazing! Please consider it!

[gunnartorfis]

I don't think I can use Remix since I need to serialize BigInt 😔

[kiliman]

Just create your own helper functions that use superjson or some other library. I wouldn't throw the baby out with the bath water.

[gunnartorfis]

If I only knew how. Would it be enough to return a promise in my loaders that resolves the data in superjson?

[sergiodxa]

In Remix Utils I have a useLoaderData/useActionData and json wrappers to let you pass custom parser and serializer functions https://github.com/sergiodxa/remix-utils#useloaderdata

[ryanwoodcox]

@kiliman After spending a while combining some custom work with the good work @sergiodxa has done with his remix-utils, I think its worth mentioning I can remove everything I've been customizing in favor of using your wonderful work in https://github.com/kiliman/remix-typedjson.

(I'm here due because of the Prisma Date serialization typing issue)

[stephen776]

Is this still being considered for a future release? Consider this another vote in favor of this working out of the box 👍

[kiliman]

superjson is kind of heavy, that's why it's not included in the core. However you can try remix-typedjson which is a fast/small (<3k) package for handling native types. I also have remix-superjson if you need the full feature set of superjson.

[stephen776]

Oh yes - I am using remix-typedjson already - it's great.

I was curious if the lighter-weight version would make it to core eventually.

I believe Next 13 has this sort of end-to-end type handling and with the rise of popularity of things like tRPC I figured it would make sense for remix to have similar support out of the box

[caprica]

I know some time has passed on this issue, but as someone new to Remix I arrived here from a search after I hit this same issue.

Is there any current recommendation for this date handling in late 2023?

I ask because I checked out remix-typedjson, remix-superjson, superjson-remix, remix-supertyped and the like, and those repos do not seem all that active, with not many recent commits, or what look like on the face of it significant open issues with the current version of Remix.

Situations like this are why it would be very helpful to have a solution for this in Remix core rather than depending on a 3rd-party repo that falls by the wayside as time goes on.

[lensbart]

It seems like remix-typedjson doesn’t provide types for route handle objects, which can depend on typeof loader, which may use typedjson from remix-typedjson. To me, it seems like this kind of stuff would make a lot of sense to be offered out-of-the-box. Do you happen to know why Remix is unlikely to support this? Thanks

[kiliman]

I'm not sure what you mean about handle objects. They are not serialized so the types will not be modified.

[sergiodxa]

I think he meant when you do something like this

export let handle = {
  something(data: SerializeFrom<typeof loader>) {
    // do something here
  }
}

And then you use useMatches to access that handle.something and call it with route.data as argument, similar to how some utils in Remix Utils works, e.g. ExternalScripts or useHydrate.

---

I doubt Remix will add support for superjson or typedjson directly because they want to remain as close to the standards as possible, that's using response.json, JSON.stringify, and JSON.parse instead of custom functions.

What I would do is to pass the data you receive on the handle object through superjson or typedjson manually, or maybe wrap this in a function that can do it for you.

[lensbart]

Thanks for the pointer, @sergiodxa. The approach you described was the one I tried initially, but I thought I was stuck. Turns out that the UseDataFunctionReturn type exposed by remix-typedjson allows me to make the types match.

I do feel like, for these cases, it would be helpful if the official Remix docs mentioned the pitfall & pointed the reader to a (list of) package(s) that can help resolve it.

[sergiodxa]

Yeah I agree on that, a docs mentioning how serialization will strip some types and how a list of libs you can use to solve it would be useful in the docs.

[thomasfr]

I have no preference for or against any existing solution, but would be awesome to have a transparent and automatic way when using useLoaderData etc. Remix already does some magic with Promises over the wire when using defer in the loader, so the Web Platform argument against this is kinda weak imo.

[sergiodxa]

With Single Fetch using turbo-stream, this is not needed anymore since now Remix can send more advanced objects like Date, Map, etc.

[jackbravo]

This should be at the top of the thread as well.

[sergiodxa]

I will update the original post to mention it