Wednesday, September 23, 2015

Jwt Authentication with Ember + SailsJs / Waterlock Part 2 - Google oAuth2

Leading on from part 1 Jwt Authentication with Ember + SailsJs / Waterlock I decided to continue with the theme by adding the ability to authenticate with a 3rd party authentication service - in this example I'm using Google oAuth2.

Now this wasn't simple for me to figure out initially! Partly because, in my mind, the confusing docs surrounding Google oAuth2 where they state:
When a user grants access to your app for a particular scope, the user is looking at the user consent screen, which includes project-level product branding that you set up in the Google Developers Console. (For information about setting up the consent screen, see Setting up OAuth 2.0 in the Developers Console help.) Therefore, Google considers that when a user has granted access to a particular scope to any client ID in a project, the grant indicates the user's trust in the whole application for that scope.
Now, I read that as: when you collect an authentication code from one medium within your applications "product" suite, provided other parts of that suite use a client ID that share the same "project" at Google's end - then you can use that client and authentication code to gain an access token - and thus validate the user. Apparently not so! I spent ages going round in hoops over this and settled on a simple setup which means I can authenticate the user on both the client (Ember) and server (Sails).

To make things clear - the idea of all this is to provide a sensible UX for a browser application to authenticate using 3rd party auth services. This includes having the auth flow inside the application and as we are an SPA we don't want to have any nasty page refreshes or redirects (in the main browser). I am pulling in a few other libraries to achieve this and have had to modify the Waterlock library slightly to enable this to work. I'll be cleaning that up and pushing back to Waterlock soon but for now I have included the deps as submodules inside our repo.

The desired flow is:

  1. User clicks "authenticate with Google" button
  2. Popup appears with Google's auth flow
  3. User provides access to our app 
  4. Popup closes (main app never refreshes or redirects)
  5. Our app receives the authorization code from Google
  6. Sends that code to our API
  7. Our Api then validates that code by exchanging it for an access token
  8. If the code is valid we find or create a user and set up the JWT for the client app
  9. Our client app receives the JWT and uses that for future requests to the server
Quite long winded but the net result is a great UX for our users and a reliable authentication flow that's secure. Now - one major change that I have had to make here to make this work - baring in mind that I have no idea how to get "Cross Client Authorization" to work - was to change the way the app access the API. To enable this I am proxying the API through ember. You can achieve this by running the following command (instead of just running "ember server"):
ember server --proxy http://localhost:1337

In terms of changes - I have included  a modified version of waterlock-google-auth in the API - this has a simple setup in /config/waterlock.js - you can see the required bits in my repo and by reading the readme on the main repo. The changes I had to implement were to ommit the requirement of the client to provide a CSRF - I may add a step where the client can retreieve one before starting the auth flow - I may omit it by using a separate endpoint for SPA auth - not sure of the implications yet. That's all for the server!

The client has a few new pieces:

Because we are customizing the standard flow a tad (sending the auth code to the API) I had to implement a custom authenticator:

The interesting part here is where I exchange the auth code for a validated user with an access token provided by our server - this makes up our JWT! Nice.

I'll likely move on to add Facebook authentication and possibly create our own Spotify add-on to Waterlock and wire that in. Finally I'm planning on adding web sockets to provide "live" UI updates.

Friday, September 18, 2015

Jwt Authentication with Ember + SailsJs / Waterlock

Thought I'd knock together a little example of using Waterlock/SailsJs with EmberJs and using JWT as the mechanism.

JSON Web Token (JWT) is a JSON-based open standard (RFC 7519) for passing claims between parties in web application environment. The tokens are designed to be compact, URL-safe and usable especially in web browser single sign-on (SSO) context.
JWT is a means of providing a simple claims based authentication between a client and a server. The token is an encrypted piece of data which can be sent as part of a payload to allow access to restricted services. The full RFC is here

The basic gist of JWT is this:

  1. User authenticates with server (identity/password, oAuth etc)
  2. Server generates JWT using secret key and some payload (usually a JSON string that provides some basic identification information, a subset of the User object for instance)
  3. Server responds with JWT access_token
  4. Client stores access_token in cookie or local storage (we will be using local storage)
  5. Client provides access_token in future requests.
While this may look a lot like the standard cookies session ID flow there is a major difference in that a JWT is a small encrypted package which defines the access claim and identifies the user. They are inherently much more secure than cookies as well as allowing statelessness - and this is the big one. Yeah sure we can manage session state across servers in a cluster using a central cache like Redis. Go ahead, hope you have a nice day breaking the fundamental design of the web. 

Waterlock is a nifty little user authentication//JSON web token management tool built for Sails:
Using these two projects in tandem makes it ridiculously easy to get up and running with JWT and a decent REST API. Couple that with EmberJs - "a framework for ambitious web applications" and you have a pretty niffty set-up.

The best way for handling auth in an Ember app is through the use of Ember Simple Auth:
While all of these parts are really simple to use they aren't necessarily that simple to get up and running with. This whole area of software development (JavaScript in particular) is evolving at a rate of knots,  JWT is a relatively new technology and server side frameworks such as SailsJs and client side frameworks like EmberJs are moving so fast it's really, really hard to keep up unless you're lucky enough to be able to dedicate your full attention! This is both good an bad news, there's a lot of well meaning blog posts out there explaining how to do XYZ with these tools but as soon as they are published they're out of date. I expect the same will unfortunately be true of this post. But for now at least the repo is using the very latest of each of the dependancies and (I hope) provides some clear guidance in getting this set up.

So that's the server side and UI components. I'm not going to go through setting up each of these components - you can see how that's all done on each of their repos. I'm also asuming that if you're interested in this little cocktail of code you're probably aware of each of them but a little confused as to how you go about pinning it all together.

So down to the code! The repo is here: go ahead and clone it if that's what flicks your switch - I expect it does else you wouldn't have read this far.

There's 2 parts to this repo:
  • The Server
  • The Client
I'll walk us through the interesting parts of the server initially - there's not much to it!

I've added a couple of extras here that help make things simple. Ember-Data is a brilliant little library - it doesn't do everything and I don't use it everywhere but where it is used it's very clever and ahead of it's time. One of those areas is being a pretty complete implementation of the JSON-API spec: And to get SailsJs dancing nicely with this spec we need to update the blueprints. To do this I use the brilliant repo by mphasize: What this does is turn the standard JSON output from sails into nice neat JSON API compatible JSON. Neat.

I've also included waterlock and waterlock-local-auth. To get all these you just need to cd into the /sonatribe-api directory and run
npm install
Now - that's pretty much it - if you're doing these from fresh you'll need to run the generate blueprints command but that's all documented on the repo's - RTFM ;)

The noteworthy config changes include in the /config/models.js file:

You can see that I've added the "associations" and "validations" nodes to the config. It just defines explicitly how you want these types of nodes to appear in the JSON output.

Next up is waterlock.js:

You're going to need to tell it to pluralizeEndpoints and also you need to rename the token name - ember-simple-auth expects "access_token" - don't do this and you're into a world of pain. You also want to set "stateless" to false - we don't want any nasty sessions on the server thank you please. Sessions are grotty at the best of time and mess with your head when you try and step up in the scalability realm. JWT is stateless authentication. It keeps things clean.

That's it for the API! Simple yeah.

So now onto the client - this is where things can get a little... involved. But considering what you're getting - I'd say not too involved. You'll want to cd into the /sonatribe-ui folder and do the usual npm install && bower install stuff.

I've included ember-cli-simple-auth and ember-cli-simple-auth-token which are set up in the /config/environment.js file:

Go with my defaults for now - you can mess with these later if you want.

Things to note:
  • I've used pod structure - it's great for organising an app small or large.
  • I've used the new computed decorators: meaning we get Java style attribute decoration for things like @observe and @property and even @whateverYouWant
All this works out of the box when you npm/bower install my repo. The login and register components can be found here - we let the framework do the work so these components are ridiculously simple:
The only noteworthy thing being the slightly different flow for registering as opposed to loging in - waterlock (currently) uses the same workflow and if an un-registered user attempts to login and "createOnNotFound" ( is true, waterlock assumes you want to create a new users - and provided the dupe checking passes (no duplicate username/email validations) - a new user is created. In future versions of Waterlock there will be an explicit /register endpoint. For now it's best to query the /users endpoint for email / username availability. I'm doing this here: (not wired up atm - I'll likely do that in the week to finish up).

So the process is to call /login if the validations for username / email pass - get the user registered and logged in and then, using the response with our authentication data we can populate the users User model and save. Finally we push the user off to the main app by sending an action to the parent route ("userAuthenticated").

I'm aware that because I am accessing Ember-Data and JQ.Ajax within this component I am completely side stepping DDAU ( - and this is a bit cringe but I'm going to leave it as is as I want the example to be clear. The simplicity of using these libraries side by side should be unhindered!

Wednesday, September 16, 2015

C# Threading Primers

I've been refreshing my brain on threading in C# recently and have compiled a list of resources I found excellent on the subject.

I'll leave them here for future me and also for anyone interested in the topic - save some google bandwidth :)

Monday, October 20, 2014

Working with .NET on ubuntu

If you're a .NET dev interested in finding out what it's like to develop .NET apps on Ubuntu the cross over can be quite painful. I've created a vagrant box which comes loaded with the bleeding edge MonoDevelop (actually compiled from the source).

To get started you need vagrant installed. You can run this on any OS: OSX, Windows or Ubuntu. You'll also need VirtualBox - both free and OS :)
Then just pull my vagrant config from the repo.

All you need to do then is open a terminal in that directory and type:

vagrant up 


It takes a while to get loaded 1st time as it builds MonoDevelop from source but it's all set up and ready to go when it's finished.

Vagrant is a great tool, lots of NET developers might not have heard of it but it makes creating dev (and prod) environments super easy.

Enjoy :)

Saturday, October 4, 2014

Hosting an ember-cli app inside a ServiceStack (or any) MVC app

The resulting code for this post can be found at:

We are looking into refactoring sonatribe to be an ember-cli SPA. Using the Discourse code base as inspiration I've been spiking various parts to see exactly how feasible this would be.

This all started with us wanting to mimic the discourse header (ignoring the messy sub header):

As you can see the header is a really nice, dock-able UI and has a lot of features. My initial thoughts after looking at the code to isolate the header were that this would be near on impossible. We wanted to do this because we want the transition from sonatribe to our discourse installation to be seamless - it's important that it feels like one app. Ignoring the obvious integration points outside of UI (things like updating sonatribe or discourse when the user receives a PM - this will come later hopefully in the form of plugins). I'd never used Ember or Ruby before and so diving into the Discourse source code was a little daunting!

Reverse engineering the code I managed to stub a lot of the code for things like preload store and bound-avatar and basically get a functioning version of the header isolated into it's own ember-cli project:

Not bad! Wiring this up to our own search API I actually managed to get the grouped search results working. You can see this working here:

The search API is a really poor knocked-together ServiceStack API which has zero optimizations so ignore the poor perf on the results!

All well and good but I was still stubbing the preload store. This is where Discourse have done the clever thing of loading the app with all of the data it needs to render itself on boot. Looking at the Discourse HTML you can see there is a lot of JS/JSON injected in. The problem here is that ember-cli doesn't really work this way. Ember-cli gives you the folder structure to develop in and uses a filesystem watcher to recompile your code (ES6, SCSS etc) into a distributable form. It even serves the index.html inside a HTTP server for you.

Discourse works this way because the SPA knows you're logged in through the presence of a user object injected into the preload store. They use omniauth which loads a popped browser window to handle the auth mechanism - once the user has authenticated it calls into a hook (window.opener.Discourse.authenticationComplete) to notify discourse that authentication has completed and sends the result. If the user successfully authenticated the browser is reloaded and because the user is authenticated the users JSON is loaded into the preload store.

So, serving the index.html as a dynamic (is that still the right term? It sounds so ASP 3.0!!!) page is pretty much a must have.

To do this we are going to need to drop the HTTP server provided by ember-cli and work out a way to push the /dist folder to another folder which can be part of a hosted project. I'm a big fan of ServiceStack but the hosted project could literally be any language/framework.

The idea here is to have 2 projects:

  1. ember-cli app where we can develop the JS and CSS
  2. an integration project where we can host index.html and inject session state and preloaded data
To keep the integration project up-to-date when we make changes in the ember-cli project we can use Bower link ( - this will replicate changes to the integration project live using symlinks. You can link the /dist folder to the target project - you can see this here:

Rebuilding the project on changes can be done using the ember build --watch command. 

What this will do is - every time you edit some code in the ember-cli project - ember will notice the files have changed and re-build the code - it then copies the resulting build to the /dist folder. Bower's symlink will then copy the code to the integration project. 

We can then host our own contents of index.html as a view inside our hosted environment: 

So now we get the mixture of serverside code inside the ember-cli index.html and we can inject in session state and preloaded data!

This means I can go back to my header spike project and replace all of the stubbed code with real user data and start to really figure out the auth aspects of this refactoring.

While this is a ridiculously simple example - the project in github does show this all working:

The "this is a test" lines are coming from the cshtml and the ember app is booted below it! All I need to do now is get the authed user from ServiceStack and serialize whatever data we need in the UI into the preload store.


Sunday, September 7, 2014

Running ServiceStack / RavenDb API on Ubuntu

I've been playing about with Docker and Vagrant a lot recently, I find them to be really amazing tools. Obviously with the amount of OS envy streaming out of my little foray with Docker in particular I am trying to move the Sonatribe API over to run on Mono inside a docker container.

ServiceStack runs great self hosted on Mono but I have been getting issues with RavenDb: async calls seem to fail and things like bulk inserts seem to fail with this:

If anyone has any help or pointers on these i'd be massively grateful!

Wednesday, July 30, 2014

Sonatribe conversations platform - CQRS knockout - part 2:

Following on from part 1 (CQRS Journey) this is my implementation spike using GetEventStore. Assesing which CQRS setup we should use at sonatribe for the new conversations feature.

The code can be found here.

This implementation is quite a bit different so I'll discuss where they branch off from each other and maybe try and figure out the pros and cons.

Using EventStore as the persistence mechanism for storing event streams takes a lot of faff out of the process. For instance if you don't want to you don't need to have any service bus sitting between the API -> domain service and read model. However - EventStore is a more lower level technology and because of that you don't get so much out of the box as you do with CQRS Journey.

With this implementation I decided to keep the domain service in a separate process at the other end of a RabbitMQ queue. So - looking at the API you won't see a huge amount of difference in terms of complexity (except maybe RMQ is heaps simpler to get up and running).

Moving on from there I have a  console app (as opposed to a worker role) running with TopShelf (for easy promotion to Windows Service when required). The code for this is here: Really simple way of wiring up the command handlers using Castle Windsor.

Looking at the one and only command handler: you can see how easy it is to save an aggregate root into the event store using the GetEventStoreRepository. This was one area that the developer is left to figure out - finding this repository was a great help.

You can see in the AR that when I create a new conversation it fires a new ConversationStarted event. This event is listened to in the read model. The wiring up for the read model you can see here: Again - there is some work to do here to turn the received events into something that can be worked with. There is also some (overly simple) code here to dispatch the events to the event handlers.

Inside the event handlers I am denormalizing the data to a RavenDb read model.

Aside from the added work of figuring out the event dispatcher I find the GES architecture much much simpler than the Azure based CQRS Journey code. There seems to be much less ceremony - even after I reduced it drastically in my refactoring of CQRS Journey. I haven't got any perf details for either implementation but Greg claims GES will natively handle 20,000 messages a second on a single node so I doubt perf is an issue for either implementation.

Next up I'll be adding some real features using each of these infrastructures. I like to work from an emergent design BDD style approach so will be looking at which of these works best under a test 1st dev cycle.