Recreating SuperTokens Middleware by Hand

Question

Mkay This kinda relates to the `Issue` I threw up on GitHub in `supertokens node` But this question is more specific Is there an existing way to replicate the ` signin` ` signup` ` session refresh` route middleware by hand And are those the only middlewares created from `app use middlewares ` ` Background in Thread `

ITEnthusiasm · Answer

Now that I m done testing and am trying to build out and deploy a physical app I m running into some unexpected troubles The `remix supertokens` example that I created works just fine However there are limitations The current implementation relies on calling `http localhost` `https localhost` If for some reason the web server decides to force `https` only then when the Remix server calls itself it must also use `https` My application is constrained in this way 1 I can t make requests to `https localhost` from my own server 2 I can make requests to `https MY DOMAIN com` from my own server 3 I don t want to make requests to `https MY DOMAIN com` from my server because it creates unnecessary round trips and will cost me more money I could go into the details of why I m in that situation but I figured that listing out the constraints was simplest Being in this conundrum I m now trying to see if there s a way to replicate the SuperTokens functionality on the server directly in my Remix Actions i e route handlers

ITEnthusiasm · Answer

Still keeping the details minimal But I m only running into this issue because I m using Cloudflare as a reverse proxy for the app and have some strict settings Calling my own server locally conflicts with my strict setup

rp · Answer

hey < ITEnthusiasm>

rp · Answer

> Is there an existing way to replicate the signin signup session refresh route middleware by hand And are those the only middlewares created from app use middlewares Yea The backend SDK has functions that are called in all these APIs that are exposed to you excepting for session refreshing I think

rp · Answer

> the web server decides to force https only You mean the cookies have the secure setting which makes it send only if using https If this is the case I think this applies only to browsers sending the cookies and calling your own API on the backend should still work If not then you could extract the cookies from the request made by the browser and then resend them in your request that uses http Im not sure i completely understand the situation

ITEnthusiasm · Answer

Ah alas If there s no way to reproduce session refreshing then I might have to rethink some things

rp · Answer

actually sorry

rp · Answer

there is refreshSession function also that is exposed

rp · Answer

so yea

rp · Answer

you can call all the functions in your APIs

rp · Answer

For example you can make your own refresh API and call this function in it https github com supertokens supertokens node blob master lib ts recipe session index ts L214

ITEnthusiasm · Answer

Yeah sorry for the confusion With Cloudflare I have their Strict Full TLS mode In this mode all communication is over TLS from the browser to Cloudflare to the origin server and all the way back Cloudflare allows you to install certificates that they trust on the origin server However they re signed by Cloudflare rather than by some other trusted CA So Node js doesn t know anything about them My current SuperTokens implementations in Remix involves Node js calling the SuperTokens endpoints on its own server But when it sees that Cloudflare signed certificates it doesn t trust them So the request fails This is why HTTP works but not HTTPS But this problem wouldn t exist if I changed to an implementation that didn t involve calling the SuperTokens endpoints from the server I m sure there are some ways to get Node to trust the Cloudflare certificates but I don t want the implementation for HTTPS to get too complicated when it comes to the security side A lot of potential places for people to get tripped up

rp · Answer

right i see Why not just make the node server call itself using localhost

ITEnthusiasm · Answer

I am That s the problem Maybe other servers act different But with Node js I have this issue If I used the domain instead `https mydomain com` then the request would go to Cloudflare not my origin server which has valid certificates But that s an unnecessary round trip that I don t want to take For clarity Cloudflare has valid CA certificates Node js trusts these Origin Server has Cloudflare signed certificates that Cloudflare trusts but that Node js does not This is typically fine except for cases where the Node js server tries to call itself directly through `localhost` Ironically

rp · Answer

oh no I mean since the server is calling itself it can just use localhost to call itself so cloudflare isn t even involved in this Am i missing something

rp · Answer

you set read cookies manually from the request response when calling localhost

ITEnthusiasm · Answer

It is already using localhost Hmmm Maybe I can try to explain things differently So imagine we re in a different scenario without Cloudflare

ITEnthusiasm · Answer

Imagine I setup a Node js Express server It uses HSTS and forces all requests to be HTTPS All requests must be HTTPS and only port `443` is exposed by the server Because of this I cannot call `http localhost` That would be an HTTP request which the server rejects Now imagine that on the server I m using a self signed certificate in the configuration Browsers do not trust this certificate Similarly no Node js application will trust this certificate If someone else tries to call my server using HTTPS e g `https mydomain com` Node js will fail the request because the certificate is not verifiable In the same way if I call `https localhost` Node js is still unable to verify the self signed certificate So even though the server is calling itself it will fail

ITEnthusiasm · Answer

If we translate that scenario over to the current one it s exactly the same But instead of using a self signed certificate I m using a Cloudflare signed certificate By default Node js trusts neither of these

rp · Answer

hmmm You could introduce nginx in front of node js that will make the https request to a http request and send it to node which is listening on http

ITEnthusiasm · Answer

I could But again that would raise complexity I d rather A Find a way to get Node js to trust Cloudflare certificates Not sure about how to go about that yet or B Implement the 3 SuperTokens API routes locally My only concern there is how simple it would be But the bright side is that the complexity is restricted to the `server js` file in that case It doesn t bleed into things like certificate authorities and such And it doesn t require something else like Nginx Unrelated This would also give me a hint to translate all my Remix SuperTokens code to a new Solid Start example I think

rp · Answer

right Fair enough Option b seems good

ITEnthusiasm · Answer

Cool I ll try to examine the docs and the SDK reference closely But I may reach out for help clarity from time to time if that s okay If I can get something working with B my hope is that it would pave the way for the other things like Svelte Kit too But I m not certain about that yet because obviously I haven t tried it all out yet

rp · Answer

sounds good yea feel free to ask

ITEnthusiasm · Answer

Mkay Either today or next Friday I m going to try to start looking into this on Fridays Need to clean up the codebase a little bit first

rp · Answer

alright

ITEnthusiasm · Answer

I m having trouble finding out where `supertokens middleware` points https github com supertokens supertokens node blob master lib ts framework express framework ts L159

ITEnthusiasm · Answer

Obviously it can t be that local function

ITEnthusiasm · Answer

Okay It s `supertokens middleware` > `recipe handleAPIRequest` it seems

ITEnthusiasm · Answer

New Question Do you know if the Node js Server Frameworks that you support e g Express Koa etc extend the existing Node js interfaces for request objects For instance these frameworks would not delete overrride `IncomingMessage headers` right

ITEnthusiasm · Answer

The typing of `any` in many places admittedly makes some aspects of looking around the code and contributing a little difficult

ITEnthusiasm · Answer

After the first couple of questions get responded to I m probably going to refer to this thread in < 757927552999227393> and then continue it there since I assume that s where this stuff really belongs

ITEnthusiasm · Answer

Okay Some more questions SuperTokens clearly creates cookies to handle authentication I only seem to see 2 cookies 1 Access Token and 2 Refresh Token Question 1 Are these the only 2 tokens that SuperTokens ever sets for its API routes Question 2 Here https github com supertokens supertokens node blob a8760b966dd1239f0e7b782b72bbfba830fcd859 lib ts recipe session recipeImplementation ts L112 On line 125 I see a function called `attachCreateOrRefreshSessionResponseToExpressRes` Is this the only function used to create the Access+Refresh tokens If the answer to `Question 2` is yes I hope it is is there a way that we can update this and the related functions to avoid manipulating the original `Response` object and to return the `ResponseHeaders` necessary for SuperTokens to properly do its auth stuff only

ITEnthusiasm · Answer

Such a solution would be framework agnostic In terms of rewriting the SuperTokens routes on my own instead of using `app use middleware ` I m finding that it would actually be a lot better if I could use a framework agnostic solution A framework agnostic solution is the only way to support the neat frameworks like SvelteKit and Solid Start right now And it will guarantee that all new hot SSR frameworks that appear in the future will immediately be supported Being able to support `next 13` when the `app` directory is out of beta would also be a big deal Of course the code would also need to be updated to accept Request information instead of the entire request object Where needed SuperTokens could probably accept a `RequestContext` `RequestData` etc object that has any needed information `path` `body` etc I m hoping that SuperTokens doesn t need that much request information to keep life simple

ITEnthusiasm · Answer

If we can get all of that going I personally feel like it would be pretty game changing It would also open the possibility of deleting `BaseRequest` `BaseResponse` and all of their child classes I know this isn t necessary and would be a breaking change but I think it would thoroughly benefit the SuperTokens team and all contributors The team wouldn t need to maintain several `abstract` classes and sub classes anymore because end users would simply interact with the `Recipe ` and `Session ` methods directly And contributors would be able to make contributions more easily I personally felt like I was hopping back and forth a lot around the codebase to get only a super basic idea of what the `signup` route does just for Express And not all types were defined That will probably be a barrier to some people who would otherwise contribute It was a barrier to me until I found out I really need this feature and SuperTokens can get a 1 Up on Auth0 with this It s a short term bother with huge long term benefits And the docs could just be updated to show how to implement the API routes in each framework I imagine it wouldn t be that hard Just spitballing more thoughts I can add this to the original issue I opened too if that would be more helpful But I think the next step would really be to add a feature for framework agnostic solutions by refactoring those underlying classes I mentioned Lmk your thoughts Sorry I know I typed a lot

rp · Answer

> Do you know if the Node js Server Frameworks that you support e g Express Koa etc extend the existing Node js interfaces for request objects Im not sure about this > Are these the only 2 tokens that SuperTokens ever sets for its API routes We also set sIdRefreshToken cookie + add front token header in the response > Is this the only function used to create the Access+Refresh tokens It doesn t create the tokens but it adds the access refresh sIdRefresh cookies and front token header to the response The name of the function is a bit misleading cause it ends with expressRes but it s also used for non express frameworks > If the answer to Question 2 is yes I hope it is is there a way that we can update this and the related functions to avoid manipulating the original Response object and to return the ResponseHeaders necessary for SuperTokens to properly do its auth stuff only Yes you can For example the createNewSession function takes in a `res` object and the `middleware` also takes in a `req` and `res` object like `middleware req res next ` Instead of giving it the actual `req` `res` objects you can instead make your own fake req res objects and pass those in Then later on you can read from the fake res object the headers cookies that were set and do whatever you like The fake `req` `res` objects would have to conform to the BaseRequest and BaseResponse shape and would need to have the boolean of `wrapperUsed` as true So even now it s technically possible to use SuperTokens with any framework via the above method You could even use this trick to convert a form post request from the frontend to a JSON request that s understandable by the supertokens API and this would make is so that in the remix backend you don t have to send a request to the backend itself which i believe you are doing now We will make this more easy document this in the future

rp · Answer

< ITEnthusiasm>

ITEnthusiasm · Answer

First thanks for reading all of that and responding Okay So 4 tokens in the header response `createNewSession` will allow me to read those tokens and add them to my own separate response object I can send my own req object in as long as it has the data needed Sound correct Is `ExpressRequest` `ExpressResponse` internal or is it exposed by the npm package I m assuming that reusing those classes would make it easier to do this If so maybe I can trying seeing if I can bring that logic to `remix supertokens` depending on how it would impact code readers I d have to see what the physical code turns out to look like first Long term though any thoughts on refactoring some of those internal functions to simply receive data and return a response headers object or something The workaround you described sounds like it should work But it will leave the SuperTokens team with maintaining some logic that s more complex and it will require end users in situations like mine to jump through more hoops It s more effort for everyone to wrap their brains around things

ITEnthusiasm · Answer

< rp> Pinging just in case the message gets missed I don t know if you have notifications turned on for this thread I think I do I d also be happy to try try to look into how much effort it would take to handle such a refactor I m less familiar with the codebase though so it would take me more time Sounds like fun though

rp · Answer

hey < ITEnthusiasm>

rp · Answer

> So 4 tokens in the header response createNewSession will allow me to read those tokens and add them to my own separate response object I can send my own req object in as long as it has the data needed Sound correct createNewSession adds theses tokens to the `res` object that you give it So you can provide a custom res object that conforms to the BaseResponse structure and then can read the tokens from that to set however you like > Is ExpressRequest ExpressResponse internal or is it exposed by the npm package It s internal > Long term though any thoughts on refactoring some of those internal functions to simply receive data and return a response headers object or something Yes > But it will leave the SuperTokens team with maintaining some logic that s more complex It won t affect us in terms of additional code maintenance since you are creating your own res req object anyway

ITEnthusiasm · Answer

> It won t affect us in terms of additional code maintenance since you are creating your own res req object anyway Sorry for the confusion I was referring to maintaining a codebase that accepts data and returns headers simpler maintenance vs a codebase that involves various subclasses of different kinds of framework Requests Responses harder maintenance I think this gives me what I need for now I ll wrap back in again if something else pops up Thanks again as always

rp · Answer

Sounds good Thanks

ITEnthusiasm · Answer

> createNewSession adds theses tokens to the res object that you give it Is there a single function that creates these tokens from the incoming request data I was hoping to rely on that But I can stop at `createNewSession` if not

ITEnthusiasm · Answer

Nevermind Not necessary As I m working through this I m trying to see if I can identify places where refactors would be simple to do in case I could actually contribute anything meaningful to the supertokens codebase myself And of course as I m thinking through these refactors I m trying to think of places ways to replace `request` inputs with simple request data inputs and replace `response` inputs with an output of response `Headers` But it seems like `createNewSession` or `attachCreateOrRefreshSessionResponseToExpressRes` would be sufficient candidates since that s where the interaction with the response is going on

ITEnthusiasm · Answer

Maybe I can wrap these guys in a function with the solution I m trying to work out in the meantime

ITEnthusiasm · Answer

< rp> ~~Question 1 I m having a bit of a hard time understanding the logic in `getCookieValueToSetInHeader` https github com supertokens supertokens node blob a8760b966dd1239f0e7b782b72bbfba830fcd859 lib ts framework utils ts L300 All of the `let` fun is making it hard to follow the logic Does this function basically set a new value for a cookie if it doesn t exist yet and replace the value for a cookie if it already exists ~~ Edit You can ignore this I m pretty sure the answer to my question is yes

ITEnthusiasm · Answer

< rp> ~~Question 2 Does SuperTokens ever try to duplicate the cookies that it sets I m actually curious to know why SuperTokens needs to filter out duplicate cookies if it s in control of when the various cookies get set already ~~ Irrelevant I m just going to include the logic to replace tokens if necessary It seems there still needs to be some standardization around this https github com whatwg fetch issues 973 but for now I have a workaround to handle this use case of replacing a key value pair for `Set Cookie` I believe

ITEnthusiasm · Answer

If you don t want me to ping you when I ask these questions lmk

ITEnthusiasm · Answer

> We also set sIdRefreshToken cookie + add front token header in the response Question 3 What does `front token` represent I ve never seen it set in my apps Is it only set when the frontend needs JavaScript

ITEnthusiasm · Answer

I m asking because I think I have `login` working now If this is good to go I think I m gonna start targetting `signup` and `refresh`

ITEnthusiasm · Answer

Mkay Sign in Create Password Reset Token and Reset Password are working now

ITEnthusiasm · Answer

Need to figure out Sign Up Session Refresh and Logout now

ITEnthusiasm · Answer

Remix code actually seems smaller with this approach At least it s hopefully more readable

rp · Answer

> What does front token represent I ve never seen it set in my apps Is it only set when the frontend needs JavaScript It s for the frontend to be able to read the access token s payload without having access to the actual access token

ITEnthusiasm · Answer

Ah okay So it does sound like it s only needed for frontend JS then Thanks

rp · Answer

Yup

ITEnthusiasm · Answer

I m seeing `sigin` `signup` `createToken` and `resetPassword` all inside `handleRequest` But I don t see anything for `logout` or `refreshToken` Do you know where this logic is defined within `supertokens node`

rp · Answer

in the session recipe

ITEnthusiasm · Answer

Question 1 What s the significance of `revokeSession` returning `true` vs `false`

ITEnthusiasm · Answer

Edit This and 4 other messages were deleted to remove clutter after I figured some things out

ITEnthusiasm · Answer

Ugh After many headaches I found out that I was unable to `logout` because the incoming `sAccessToken` that s needed for the initial `Session getSession` call appears to be URI encoded

ITEnthusiasm · Answer

Question 2 Does the `SuperTokens` ` signout` route automatically run `decodeURIComponent` on the `sAccessToken` I didn t see any logic doing this on the `supertokens node` repo but decoding the token seems to be required for the code to work

ITEnthusiasm · Answer

And I get this when trying to refresh sessions > SuperTokens core threw an error for a POST request to path recipe session refresh with status code 400 and message Field name antiCsrfToken is invalid in JSON input

ITEnthusiasm · Answer

Ughhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh This refresh session was failing because `headers get anti csrf ` was returning `null` instead of `undefined` Changing to `headers get anti csrf undefined` makes the code work Question 3 What s causing that to break Can SuperTokens not work with both `undefined` and `null` here

ITEnthusiasm · Answer

I m almost done I think I ll try to implement `signUp` tomorrow But after that I d appreciate some review to make sure the things that I ve written actually work as expected if that wouldn t be a bother similar to what we did the first time I created `remix supertokens` Assuming I can get `signUp` working I still don t know if I d say SuperTokens supports things like SolidStart and SvelteKit Once the developer needs knowledge of internals to get things working it s no longer supported But I m at least glad there s an option to cleverly interact with the internals

ITEnthusiasm · Answer

Question 4 If I want to do email verification is that logic that I d have to put inside my `signUp` function as well when I create it Or does `EmailPassword signUp` automatically take care of that for me

rp · Answer

> Question 1 What s the significance of revokeSession returning true vs false It returns true if the session existed and was revoked If the session did not exist in the first place it returns false > Question 2 Does the SuperTokens signout route automatically run decodeURIComponent on the sAccessToken I didn t see any logic doing this on the supertokens node repo but decoding the token seems to be required for the code to work Not really The token shouldn t be encoded at all just directly pass to getSession > What s causing that to break Can SuperTokens not work with both undefined and null here It can but we didn t need it > Question 4 If I want to do email verification is that logic that I d have to put inside my signUp function as well when I create it Or does EmailPassword signUp automatically take care of that for me Email verification is a different recipe and independent to emailpassword recipe Check that out

ITEnthusiasm · Answer

> Not really The token shouldn t be encoded at all just directly pass to getSession Interesting The encoding might be happening somewhere in Express or Remix then In any case I can decode the value to make things work properly I ll have to take note of this for the future

ITEnthusiasm · Answer

Sorry I thought I would get the example out today but I didn t I ll let you know when I actually have it out there I have the rough draft finished on my local now I think I m going to give it some review either tomorrow or the day after Then I ll ping you

rp · Answer

Sounds good

ITEnthusiasm · Answer

sigh okay

ITEnthusiasm · Answer

< rp> You can look at the `experiment custom supertokens` branch of the `remix supertokens` repo to see the latest code https github com ITenthusiasm remix supertokens tree experiment custom supertokens The most important changes at least right now are in this last commit if you want to look at diffs https github com ITenthusiasm remix supertokens commit f50eabc105885ddd024730f1f1652fa741204e23 If you and or any other members could review the code and test out the app to make sure everything is working correctly that would be appreciated Then if you guys think it s a good idea I can merge this into `main` on the `remix supertokens` repo My hesitation is that there s a little more code that devs have to write to get their `Remix` apps running But people using `Remix` in certain ways will end up forced to use this approach anyway I d especially appreciate it if you could let me know if there are any error cases that I m failing to handle properly especially around the ` logout` and ` auth session refresh` routes Thanks again for all your help in this This thread is several comments long If this works out I m probably going to do some minor cleanup here or there Then after I take a break I ll probably see if I can tackle `SvelteKit` with this approach maybe But I m only intending for these to be temporary workarounds since these solutions make assumptions about internal logic within `supertokens node` It s a little dangerous In terms of the `supertokens node` refactors that could be done maybe this code could give ideas about the potential inputs outputs to require idk But from this experiment it seems like the `supertokens node` functions methods really only need request `Headers` as inputs instead of the whole request object and it seems like `supertokens node` can return response `Headers` instead of accepting and mutating a response object

ITEnthusiasm · Answer

Kinda funny Because I needed SuperTokens I left SvelteKit for Remix Fast forward a couple months and now maybe this will provide a way to get SvelteKit up and running Even so I m probably going to stick to Remix

ITEnthusiasm · Answer

I m not sure if the findings here influence this conversation at all https discord com channels 603466164219281420 757927552999227393 1001934155199815730 Maybe they don t I m running with `Remix + Express` so I m not sure what the Cloudflare limitations would be

ITEnthusiasm · Answer

Question What does `getAllCORSHeaders` return Is there a place where we can see the derivation logic All the static methods I see on the repo just return empty arrays

ITEnthusiasm · Answer

Just as a heads up on that timeline of cleanup update merge branches SvelteKit etc that probably won t start until the Friday after this coming Friday I originally intended to only work on this on Fridays but obviously a bit of speedrunning happened this week I m gonna take a tiny break and then hop back into Each Friday afterwards

rp · Answer

> What does getAllCORSHeaders return See the session recipe

rp · Answer

hey < ITEnthusiasm> thanks a lot for all the work so far I ll checkout the repo sometime this week

rp · Answer

Sorry sometime next week

ITEnthusiasm · Answer

Sounds good Thanks

rp · Answer

Thank you sir

ITEnthusiasm · Answer

Looks like `next 13` is also gradually adopting support for web standard `Request` `Response` If they do this all the way this approach will also work in Next js without any additional refactoring Refactoring the approach to work with Next js would be rather easy to do anyway though Really glad Remix came along It definitely pushed Next js to make some better changes

rp · Answer

Yea This does seem to be getting more popular indeed

ITEnthusiasm · Answer

> Not really The token shouldn t be encoded at all just directly pass to getSession Something I just discovered that s probably worth noting from the `cookie` package docs https github com jshttp cookie cookieserializename value options For the `encode` option passed to `cookie serialize` > Specifies a function that will be used to encode a cookie s value Since value of a cookie has a limited character set and must be a simple string this function can be used to encode a value into a string suited for a cookie s value > > The default function is the global `encodeURIComponent` which will encode a JavaScript string into UTF 8 byte sequences and then URL encode any that fall outside of the cookie range Similarly for the `decode` option passed to `cookie parse` > Specifies a function that will be used to decode a cookie s value Since the value of a cookie has a limited character set and must be a simple string this function can be used to decode a previously encoded cookie value into a JavaScript string or other object > > The default function is the global `decodeURIComponent` which will decode any URL encoded sequences into their byte representations > > note if an error is thrown from this function the original non decoded cookie value will be returned as the cookie s value So it seems the `cookie` package that `supertokens node` uses automatically encodes and decodes That s why I ran into that hiccup earlier

rp · Answer

Hmmm Thats interesting Cause if we take the cookie value from the browser copy from network tab the value is not encoded in any way

rp · Answer

but i guess the browser must be decoding it before showing it

ITEnthusiasm · Answer

Yeah I think so When I get the cookie back from the browser on the server side it appears encoded But that would be consistent with what the package says

ITEnthusiasm · Answer

I guess it makes sense for the network tab to decode for readers

rp · Answer

Sounds about right Thanks

rp · Answer

I have planned sometime this week to checkout the remix app btw

rp · Answer

will get back

ITEnthusiasm · Answer

Sounds good Thanks

ITEnthusiasm · Answer

Added 2 new commits to this branch in order to make things easier in Svelte Kit land If you feel like reading you can find the reasoning and the physical changes at https github com ITenthusiasm remix supertokens pull 5 The changes weren t huge I ve discovered that with this new approach I can indeed use Svelte Kit With the previous implementation I could not do so easily due to the odd way Svelte Kit is structured At least I can successfully login logout refresh without JS The code is rough though of course and the Remix app isn t fully translated over to Svelte Kit yet But when everything is translated over I might make the repo public Well I ll wait to make the repo public until we know everything is properly settled in Remix Land Then I ll make the Svelte Kit repo public There s fair reason to assume that if these two SSR frameworks work then SolidStart should also be fine Though I ll again reiterate that this solution is a temporary workaround

rp · Answer

Sounds good Thanks I ll see the repo right now

rp · Answer

I saw the code < ITEnthusiasm> and mostly it s on the right track The one issue is that calls to getSession can throw a try refresh token or unauthorised exception which need to be handled Try refresh token exception should redirect the user to the refresh route and back This not only applies to the sign out route but any route that uses the getSession function Im trying to think if the middleware can be used instead of having to recreate all the routes And i think this may work You want to call the middleware function on all handlers or maybe on a global handler something like this ```ts export default function handleRequest request Request responseStatusCode number responseHeaders Headers remixContext EntryContext requestWrapper = createSuperTokensRequestWrapper request responseWrapper = createNewSuperTokensResponse callbackCalled = false await middleware requestWrapper responseWrapper => this means that the middleware did not handle the request callbackCalled = true if callbackCalled this means that the middleware didn t handle it so we let other routes handle it let markup = renderToString responseHeaders set Content Type text html return new Response < DOCTYPE html> + markup status responseStatusCode headers responseHeaders else this means that the middleware did handle the request TODO read from responseWrapper and send relevant response ```

rp · Answer

The `createSuperTokensRequestWrapper` and `createNewSuperTokensResponse` would be similar to what you have already made except that it would also do a mapping of input request paths with what the middleware expects For example for the refresh API the middleware expects it to be a POST request to ` auth session refresh` So in the `getMethod` function of the request wrapper you could detect that if the actual request path is ` auth session refresh` GET return a POST instead so that the middleware handles it About the ` TODO read from responseWrapper and send relevant response ` part you would have to create mapping from responses written by the middleware to html text responses that you want to send to the frontend If the response contains a 401 status code without clearing the cookies it means you want to navigate to the refresh route else to the login page So essentially you would end up reusing all the middleware logic making the overall code easier If this kind of makes sense I can attempt to try this method out by forking your repo If there is fundamental issue with this im not too familiar with remix lmk

ITEnthusiasm · Answer

> The one issue is that calls to getSession can throw a try refresh token or unauthorised exception which need to be handled Yes Currently the `server js` file handles the error cases for `getSession` I think You can look at the `deriveSession` and `setupRemixContext` functions to verify The `server js` file is the entry point for the entire Remix app so any session errors that would have required redirects would already have been caught before I attempt to call my custom logout and refresh functions Was that the only concern in terms of functionality Does everything else seem to be working

ITEnthusiasm · Answer

> Im trying to think if the middleware can be used instead of having to recreate all the routes Unfortunately I cannot use middleware This goes back to https github com supertokens supertokens node issues 438 though some things in that issue could probably be updated The emerging SSR frameworks are highly incompatible with the concept of middleware And although I previously forced something to work in Remix I couldn t use that trick anymore when I needed HTTPS Some comments about that are in this Discord thread Although I ve already created an issue I can try to more precisely list out the issues in an RFC or something This is why I made these helper functions e g `SuperTokensHelpers signin` And in reality these helper functions are pretty small and simple Oftentimes developers will make small simple functions in order to handle interactions with their databases this is to be expected for SSR frameworks apps With something as important as authentication it s not surprising nor is it inconvenient for developers to also write a small amount of code to make sure auth is working correctly And it s more flexible for developers to do this in their framework s way than it is to try to push it into their middleware if their framework even allows that

ITEnthusiasm · Answer

Although this code was intended to make sure that people including myself could reliably use Remix + HTTPS it was also intended to be a POC to give ideas on how `supertokens node` could be refactored such that the `request` and the `response` are not interacted with nor required directly Instead of interacting with `req` `res` the functions methods would take in cookie data and return response headers + response cookies If that happened then in my codebase the `SuperTokensData Input` and `SuperTokensData Output` classes would go away And I as well as all other developers would only need to work with `SuperTokensHelpers` in a single file In fact the helpers could be inlined directly in the server `action`s if desired This seems to provide a simple but flexible approach for developers whether they use an SSR framework or `Express` And from the looks of what I have the refactors would really only be needed for `Session` s methods `getSession` `createSession` etc

ITEnthusiasm · Answer

> So essentially you would end up reusing all the middleware logic making the overall code easier Easier in one sense but also more difficult in another Although I and other developers would get to write less code the code would be much more confusing to read or reason about and a developer would have much less insight into what s actually going on An ideal codebase keeps the code as small and simple as possible while also communicating what s happening as extensively as possible This is a hard but necessary balance to strike Rich Harris creator of Svelte + Svelte Kit has commented on the significance of this before https github com sveltejs kit issues 334 issuecomment 806217369 But beyond this I sadly cannot use middleware as previously mentioned

ITEnthusiasm · Answer

Ah I misread My apologies For some reason I thought you were saying a `handleRequest` function would get passed to an Express Middleware This is `entry server tsx` which is different Edit The snippet doesn t work ~~I suppose that could potentially work but that still risks the code requiring a bit of hoop jumping for things to work as expected And this would result in cognitive overhead for developers An implementation would also be necessary for Remix s data requests It also means that developers would rely on the internal logic of SuperTokens which is never a good thing And I think a better experience can be served to developers than giving them wrappers that still express reliance on internals ~~ To me it s still better to express functionality explicitly in simple functions as I mentioned in the comment chain earlier And as far as `supertokens node` goes one of the best solutions I think is an update to `Session` Otherwise the code would still be a bit confusing and `supertokens node` would have to create custom wrappers for every single SSR framework which is impractical

ITEnthusiasm · Answer

Update I actually think the code snippet you shared wouldn t work sadly `handleRequest` in `Remix` is only for server rendering it doesn t interact at all with the `loader`s `action`s etc And it s the `action`s and `loader`s that specifically need to be called during login logout refresh etc Moreover `handleDataRequest` separate from `handleRequest` only allows us to interact with the result of a `loader` or `action` it won t override any logic for us This of course makes sense because if there s any action related logic it should simply be placed directly in the action So we cannot reuse `middleware ` anyway The other SSR frameworks Svelte Kit SolidStart etc will run into a similar issue I really don t think there s a way around updating `Session` s methods if there s a desire to support all popular frameworks in a flexible reliable way

rp · Answer

> Currently the server js file handles the error cases for getSession I think Ah i see Makes sense But I also see getSession being used in `SuperTokensHelpers logout` function how will errors from getSession be caught from this function Also if you are already doing session verification in server js why also do it in SuperTokensHelpers logout The getSession in server js in `deriveSession` should probably pass in the sessionRequired option as `false` This will prevent it from throwing an unauthorised error and return `undefined` instead of the session object If it returns `undefined` the value of `res locals` would be `res locals = user id undefined ` Why are we doing `getLoadContext => res locals ` in the `createRequestHandler` function call if we are not consuming it anywhere Maybe I just can t find where it is being consumed About not using the middleware Im really not sure if thats the best approach from SuperTokens point of view cause whilst for simple use cases expecting devs to write the API logic themselves using the helper functions works But when we add multi tenancy account linking etc to the mix it suddenly get s really complex Let me ask you this If the APIs exposed via the supertokens middleware were exposed via the supertokens core itself similar to other auth services how would those APIs be called from the remix frontend Cause technically you could use the supertokens middleware in server js as a regular express middleware and then from remix s point of view it would just be APIs exposed by some auth service that happens to have the same domain which it needs to call

rp · Answer

I am unable to start the remix app I did ``` npm i d npm run dev ``` Got the following error https gist github com rishabhpoddar bfce3750a10fd373562d5e137c861d24

rp · Answer

I made a small demo app showing how we can use the middleware provided by supertokens + do session verification and refreshing without manually hacking the req res object https github com rishabhpoddar remix supertokens This demo app also uses express to serve the remix app

ITEnthusiasm · Answer

> How will errors from getSession be caught from this function That s where I was asking if there were any special error cases that I should handle Since the `server js` file handles the authentication piece I was hoping that any errors would be caught But I figured you guys would know for sure > Also if you are already doing session verification in server js why also do it in SuperTokensHelpers logout I need something that will allow me to revoke the current session and get response cookies to send back to the browser That was the only approach I was aware of Same for the token refresh > The getSession in server js in deriveSession should probably pass in the sessionRequired option as false For some reason I still seemed to be getting errors even when I made `sessionRequired` false so I didn t bother Maybe I can try again > Why are we doing getLoadContext => res locals in the createRequestHandler function call if we are not consuming it anywhere `getLoadContext` is what sets the `context` object that s passed to all of Remix s server functions i e `loaders` `actions` etc The `server js` file doesn t do anything with `res locals` Remix does

ITEnthusiasm · Answer

Yes I believe this happened before when you tested the older version of the Remix SuperTokens repo The `README md` file includes information about how to get the application started as well as other potentially useful details In order for the app to work you first have to compile the SCSS files to CSS using `npm run sass` You don t have to keep the sass compiler running if you don t want to After that the configuration variables need to be set properly through the ` env` file The variable names can also be found in the `README md` file

ITEnthusiasm · Answer

> Let me ask you this If the APIs exposed via the supertokens middleware were exposed via the supertokens core itself similar to other auth services how would those APIs be called from the remix frontend As we discussed earlier https discord com channels 603466164219281420 1049748644837990451 1049748718838108210 using a middleware based approach is unfortunately not a viable solution for Remix if the server requires HTTPS The repo that you shared is more or less what I already have at https github com ITenthusiasm remix supertokens the `main` branch But that implementation breaks when using HTTPS so it is no longer valid Even if middleware worked here it would not be guaranteed to work for `Svelte Kit` or `SolidStart` I suppose that theoretically the Remix application would be able to work if it called SuperTokens core directly We can t call `http localhost 3000` when the server is HTTPS only this is a completely valid use case And we can t call `https localhost 3000` because it s impossible to make secure valid certificates for localhost Using something like `mkcert` in prod would be an unnecessary dangerous and hacky solution We shouldn t and sometimes can t call `https MYDOMAIN com` because it creates an unnecessary round trip that hurts user experience waiting time For highly active applications this is impractical It s also just hacky We can call `http localhost 3567` because it is not an HTTPS only server and does not require round trips So that s good However taking this approach would require developers to learn the core API And that seems a lot more complicated than just using an updating version of `Session`

ITEnthusiasm · Answer

> About not using the middleware Im really not sure if thats the best approach from SuperTokens point of view cause whilst for simple use cases expecting devs to write the API logic themselves using the helper functions works But when we add multi tenancy account linking etc to the mix it suddenly get s really complex I obviously don t know all the ins and outs of `supertokens node` But to me the solution to this problem is pretty simple Improve the APIs to simplify the developer experience Just like I m doing the on the experimental branch of my Remix app `supertokens node` could expose helper functions for the more complex use cases And these helper functions can still accept the input headers cookies as a `Map` and return the response headers and cookies as separate `Map`s as shown on the branch And as long as the documentation was kept crystal clear and the `supertokens node` helper functions simplified and accomplished as much as they could without abstracting too much away e g without requiring `req` `res` objects the developer experience would still be good

ITEnthusiasm · Answer

From how `Remix` s approach to session management looks https remix run docs en v1 utils sessions using sessions and from how `SolidStart` s approach to session management looks https start solidjs com advanced session and from Rich Harris s creator of Svelte + Svelte Kit expressed concerns about middleware and his thoughts expectations regarding the future https github com sveltejs kit issues 334 issuecomment 804987028 I really think these SSR frameworks are moving away from middleware and towards a clearer more functional more explicit approach I want `supertokens node` to be able to push into these frameworks with the advantage of flexibility But it s going to be harder to do this if `supertokens node` goes against the flow of what these SSR frameworks are trying to do and what they require allow and ban Long term I m not saying that `supertokens node` needs to throw away middleware But I am saying that the core pieces of SuperTokens e g the `Session` class shouldn t assume that they re in a middleware like context If they do the userbase will be limited and SuperTokens will run into these incompatibility issues And if for any reason another competitor arises that is more compatible or an existing competitor catches on and changes their approach then SuperTokens will be at a disadvantage

rp · Answer

Think of the middleware as just a way to expose auth APIs that would have otherwise been exposed via the supertokens core

rp · Answer

The stuff that deeply ties with your application framework is the session management piece

rp · Answer

For that we are planning on adding functions which just take in tokens and return tokens This will work without request response objects thanks to our conversations

rp · Answer

But the middleware would stay cause really some of the api logic will get quite complex with new features But we would also expose all the raw functions that you can use in your own APIs without the middleware

ITEnthusiasm · Answer

Yes I wasn t saying that the middleware needs to be deleted though perhaps they could be more easily re used across frameworks express koa etc not Remix SolidStart if they were updated to use the new token based functions Less need for things like `BaseResponse` That s just a random separate comment though I m not saying that s a crazy priority or anything But as long as there s a way to push forward without requiring middleware I think that would be huge so thanks

rp · Answer

Yea There would be Just that it would sometimes be a pain for you to correctly replicate some of the api logic that the middleware provides But definitely possible

ITEnthusiasm · Answer

> Think of the middleware as just a way to expose auth APIs that would have otherwise been exposed via the supertokens core Yes The middleware concept itself makes sense From what I can tell the middleware simplifies interaction with the core noticeably It s just that for the special cases like HTTPS only middleware gets difficult I guess for Remix I could spin up a separate http server on a different port ```ts const remixApp = express const superTokens = express remixApp listen 443 superTokens listen 3458 ``` But that still doesn t guarantee anything for Svelte Kit or SolidStart or other future frameworks

rp · Answer

Could you explain the issue with https only

ITEnthusiasm · Answer

Yeah I m hoping that at least things will be straightforward for more common use cases especially with help from the docs But I guess we ll see as that plays out

ITEnthusiasm · Answer

If a server requires HTTPS then all requests to the server must be HTTPS This includes instances when the server calls itself However valid certificates can t be created for `localhost` So when someone attempts `fetch https localhost 443 ` etc the request fails because Node js doesn t trust the server even though it s calling itself

rp · Answer

Ah right I see But http localhost would work just fine as well

ITEnthusiasm · Answer

Exactly But `http` is banned on https only servers

ITEnthusiasm · Answer

The code snippet I gave earlier would technically circumvent the problem The separate server would just be http so Node js doesn t have to verify any certificates But again that s only in Remix and only in the use case where the developer is using the node adapter

rp · Answer

Right What server is https only though

ITEnthusiasm · Answer

In my example app Nothing My example app just uses http However the separate project that I m working on with SuperTokens only allows https It only listens on `443` and has some higher security configurations through its integration with Cloudflare

rp · Answer

Hmmm Forcing https for localhost is a little odd But I see

rp · Answer

I m not sure how common of a setup is that I totally get that https would be forced for non localhost urls even for staging env but for localhost forcing https may be uncommon

ITEnthusiasm · Answer

Well it s a full stack app on one sever So everything goes through that remix server And the problem is that I want to make sure all users are using HTTPS The cascading effect of that is that calls to my own server are also forced to be HTTPS But that isn t an intentional design decision It just comes with the territory

ITEnthusiasm · Answer

Again if I spun up a separate express server in the same `server js` file and attached SuperTokens middleware to that then I could make that an http server Then the problem goes away as far as my own project is concerned But that won t be a valid solution for people who aren t using the Node Adapter for Remix And it s not reliable for other SSR frameworks

rp · Answer

Right yea That s true Agreed

ITEnthusiasm · Answer

Yeah Coding is hard lol

rp · Answer

Haha Lots of possibilities

ITEnthusiasm · Answer

I just want to express again that I really appreciate you engaging all of these extended conversations and design thoughts I m sure it takes an extended period of time to read and respond to all these things and I know I write a lot I am genuinely interested in the well being of ST though I hope these things aren t a bother Though if they ever are you and the team are free to say so

rp · Answer

I appreciate these conversations too It helps us a lot Thank you and would love to keep exploring ideas

rp · Answer

Based on these conversations we decided to add versions for getSession and refreshSession that do not rely on req response objects Just tokens in and out It s up to the dev on how to add the tokens to the actual response cookies

rp · Answer

I believe that this would make at least some part of your approach easier

ITEnthusiasm · Answer

I believe so as well Looking forward to the updates If I ever get any other ideas I guess I ll spin those off separately But I probably won t run into them until I start trying to expand the capabilities of the existing `remix supertokens` repo I d like to add other kinds of examples to it but I just haven t had the time

ITEnthusiasm · Answer

I guess if that s the path forward I may just merge this new code into `main` after tagging what I currently have on there Then I ll update and push `svelte kit supertokens` and `solid start supertokens`

ITEnthusiasm · Answer

as I have time hopefully not long lol

rp · Answer

Sounds good You may also want to take a look at the demo app I made and see the parts specific to server js where I call getSession revokeSession and refreshSession That makes it so that your remix code loaders actions etc don t actually need to call these functions in them ever

rp · Answer

Which actually might make the whole getSession without req res not required Lol

ITEnthusiasm · Answer

Well just for Remix Not for the other guys But yes

rp · Answer

Yup

ITEnthusiasm · Answer

Though I will say I think that it gives an easier time to the readers when they can just stay within the `app` directory When someone sees a ` login` route and a ` reset password` route they re probably going to be thinking Where are ` logout tsx` and ` refresh tsx` And there will be cognitive overhead with that So I was trying to circumvent that as far as the `remix supertokens` example is concerned People in these SSR frameworks are trained to put as much into the app main project directory as possible for ease of use

rp · Answer

I see Well this is a personal choice Anyway we will add those functions which don t depend on the req res objects

ITEnthusiasm · Answer

Oh Can you ping me whenever the update happens so I can update my repos

rp · Answer

Sure We will probably also announce it in the new release channel anyway

ITEnthusiasm · Answer

Finally have Svelte Kit going with SuperTokens using the same approach as in the Remix repo Hopefully it should be as simple as `npm install > npm run dev` Simpler startup than the Remix app Still needs a custom ` env` file though https github com ITenthusiasm svelte kit supertokens

ITEnthusiasm · Answer

It s come full circle from me coming here and asking if there was a Svelte Kit integration to me leaving Svelte Kit for Remix to Remix + SuperTokens to Remix + SuperTokens + HTTPS to Svelte Kit with SuperTokens Unfortunately I m not going to refactor my project from Remix back to Svelte Kit though In any case other people should be able to reference that repo in the meantime if they re die hard Svelte fans

ITEnthusiasm · Answer

It seems like `SolidStart` still has some server bugs to iron out So I haven t been able to make any additional progress there I really wanna use `SolidStart` though I would translate my other project from Remix to SolidStart if it was an option

ITEnthusiasm · Answer

There are technically a couple things that could be made cleaner with this repo But overall it s sufficiently functional I think I guess you can verify if it actually functions correctly when you get the opportunity if you want

rp · Answer

sounds good thanks < ITEnthusiasm> i ll check it out sometime next week

ITEnthusiasm · Answer

No rush by any means Did you get to check out the Svelte repo

rp · Answer

not yet < ITEnthusiasm> will do it very soon though I have it in my todos

ITEnthusiasm · Answer

Sounds good

ITEnthusiasm · Answer

< rp> It seems `createNewSession` now requires a request object in order to function correctly This seems to be old news now https github com supertokens supertokens node blob master CHANGELOG md 1300 2023 02 01 https github com ITenthusiasm svelte kit supertokens issues 1 This adds slightly extra effort for SvelteKit Remix etc as now I need to update the example repos Does anything in particular need to be tacked onto this request object for the session creation to work correctly Or would a random request object without any meaningful information be sufficient I was hoping things would move more towards data in data out than req res in req res out Was there a motivation for this change

rp · Answer

We are working on a version of createNewSession getSession and refreshSession that don t depend on the req res object at all This should be out fairly soon

rp · Answer

you should wait We will be releasing an update fairly soon

ITEnthusiasm · Answer

Okay will do Thanks again I feel like it s been a while since we ve had any back and forth Been busy Hope things are well for you guys

rp · Answer

Yea Thanks Hope things are well for you too