Create assert_response_schema test helper

It is a common pattern to use JSON Schema to validate a API response[1], [2]
and [3].

This patch creates the `assert_response_schema` test helper that helps people do
this kind of validation easily on the controller tests.

[1]: https://robots.thoughtbot.com/validating-json-schemas-with-an-rspec-matcher
[2]: https://github.com/sharethrough/json-schema-rspec
[3]: https://github.com/rails-api/active_model_serializers/issues/1011#issuecomment-127608121

CHANGELOG.md

@@ -16,6 +16,8 @@ Breaking changes:
 
 Features:
 
+- [#1270](https://github.com/rails-api/active_model_serializers/pull/1270) Adds `assert_response_schema` test helper (@maurogeorge)
+- [#1099](https://github.com/rails-api/active_model_serializers/pull/1099) Adds `assert_serializer` test helper (@maurogeorge)
 - [#1403](https://github.com/rails-api/active_model_serializers/pull/1403) Add support for if/unless on attributes/associations (@beauby)
 - [#1248](https://github.com/rails-api/active_model_serializers/pull/1248) Experimental: Add support for JSON API deserialization (@beauby)
 - [#1378](https://github.com/rails-api/active_model_serializers/pull/1378) Change association blocks
@@ -45,7 +47,6 @@ Features:
     CollectionSerializer for clarity, add ActiveModelSerializers.config.collection_serializer (@bf4)
 - [#1295](https://github.com/rails-api/active_model_serializers/pull/1295) Add config `serializer_lookup_enabled` that,
   when disabled, requires serializers to explicitly specified. (@trek)
-- [#1099](https://github.com/rails-api/active_model_serializers/pull/1099) Adds `assert_serializer` test helper (@maurogeorge)
 
 Fixes:
 - [#1239](https://github.com/rails-api/active_model_serializers/pull/1239) Fix duplicates in JSON API compound documents (@beauby)

active_model_serializers.gemspec

@@ -54,4 +54,5 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency 'timecop', '~> 0.7'
   spec.add_development_dependency 'minitest-reporters'
   spec.add_development_dependency 'grape', ['>= 0.13', '< 1.0']
+  spec.add_development_dependency 'json_schema'
 end

docs/howto/test.md

@@ -1,6 +1,6 @@
 # How to test
 
-## Test helpers
+## Controller Serializer Usage
 
 ActiveModelSerializers provides a `assert_serializer` method to be used on your controller tests to
 assert that a specific serializer was used.
@@ -16,3 +16,137 @@ end
 
 See [ActiveModelSerializers::Test::Serializer](../../lib/active_model_serializers/test/serializer.rb)
 for more examples and documentation.
+
+## Serialization against a schema
+
+### Dependencies
+
+To use the `assert_response_schema` you need to have the
+[`json_schema`](https://github.com/brandur/json_schema) on your Gemfile. Please
+add it to your Gemfile and run `$ bundle install`.
+
+### Minitest test helpers
+
+ActiveModelSerializers provides a `assert_response_schema` method to be used on your controller tests to
+assert the response against a [JSON Schema](http://json-schema.org/). Let's take
+a look in an example.
+
+```ruby
+class PostsController < ApplicationController
+  def show
+    @post = Post.find(params[:id])
+
+    render json: @post
+  end
+end
+```
+
+To test the `posts#show` response of this controller we need to create a file
+named `test/support/schemas/posts/show.json`. The helper uses a naming convention
+to locate the file.
+
+This file is a JSON Schema representation of our response.
+
+```json
+{
+  "properties": {
+    "title" : { "type" : "string" },
+    "content" : { "type" : "string" }
+  }
+}
+```
+
+With all in place we can go to our test and use the helper.
+
+```ruby
+class PostsControllerTest < ActionController::TestCase
+  test "should render right response" do
+    get :index
+    assert_response_schema
+  end
+end
+```
+
+#### Load a custom schema
+
+If we need to use another schema, for example when we have a namespaced API that
+shows the same response, we can pass the path of the schema.
+
+```ruby
+module V1
+  class PostsController < ApplicationController
+    def show
+      @post = Post.find(params[:id])
+
+      render json: @post
+    end
+  end
+end
+```
+
+```ruby
+class V1::PostsControllerTest < ActionController::TestCase
+  test "should render right response" do
+    get :index
+    assert_response_schema('posts/show.json')
+  end
+end
+```
+
+#### Change the schema path
+
+By default all schemas are created at `test/support/schemas`. If we are using
+RSpec for example we can change this to `spec/support/schemas` defining the
+default schema path in an initializer.
+
+```ruby
+ActiveModelSerializers.config.schema_path = 'spec/support/schemas'
+```
+
+#### Using with the Heroku’s JSON Schema-based tools
+
+To use the test helper with the [prmd](https://github.com/interagent/prmd) and
+[committee](https://github.com/interagent/committee).
+
+We need to change the schema path to the recommended by prmd:
+
+```ruby
+ActiveModelSerializers.config.schema_path = 'docs/schema/schemata'
+```
+
+We also need to structure our schemata according to Heroku's conventions
+(e.g. including
+[required metadata](https://github.com/interagent/prmd/blob/master/docs/schemata.md#meta-data)
+and [links](https://github.com/interagent/prmd/blob/master/docs/schemata.md#links).
+
+#### JSON Pointers
+
+If we plan to use [JSON
+Pointers](http://spacetelescope.github.io/understanding-json-schema/UnderstandingJSONSchema.pdf) we need to define the `id` attribute on the schema. Example:
+
+```js
+// attributes.json
+
+{
+  "id": "file://attributes.json#",
+  "properties": {
+    "name" : { "type" : "string" },
+    "description" : { "type" : "string" }
+  }
+}
+```
+
+```js
+// show.json
+
+{
+  "properties": {
+    "name": {
+      "$ref": "file://attributes.json#/properties/name"
+    },
+    "description": {
+      "$ref": "file://attributes.json#/properties/description"
+    }
+  }
+}
+```

lib/active_model/serializer/configuration.rb

@@ -21,6 +21,7 @@ module ActiveModel
 
         config.adapter = :attributes
         config.jsonapi_resource_type = :plural
+        config.schema_path = 'test/support/schemas'
       end
     end
   end

lib/active_model/serializer/railtie.rb

@@ -21,6 +21,7 @@ module ActiveModel
     end
 
     if Rails.env.test?
+      ActionController::TestCase.send(:include, ActiveModelSerializers::Test::Schema)
       ActionController::TestCase.send(:include, ActiveModelSerializers::Test::Serializer)
     end
   end

lib/active_model_serializers/test.rb

@@ -2,5 +2,6 @@ module ActiveModelSerializers
   module Test
     extend ActiveSupport::Autoload
     autoload :Serializer
+    autoload :Schema
   end
 end

lib/active_model_serializers/test/schema.rb

@@ -0,0 +1,103 @@
+module ActiveModelSerializers
+  module Test
+    module Schema
+      # A Minitest Assertion that test the response is valid against a schema.
+      # @params schema_path [String] a custom schema path
+      # @params message [String] a custom error message
+      # @return [Boolean] true when the response is valid
+      # @return [Minitest::Assertion] when the response is invalid
+      # @example
+      #   get :index
+      #   assert_response_schema
+      def assert_response_schema(schema_path = nil, message = nil)
+        matcher = AssertResponseSchema.new(schema_path, response, message)
+        assert(matcher.call, matcher.message)
+      end
+
+      MissingSchema = Class.new(Errno::ENOENT)
+      InvalidSchemaError = Class.new(StandardError)
+
+      class AssertResponseSchema
+        attr_reader :schema_path, :response, :message
+
+        def initialize(schema_path, response, message)
+          require_json_schema!
+          @response = response
+          @schema_path = schema_path || schema_path_default
+          @message = message
+          @document_store = JsonSchema::DocumentStore.new
+          add_schema_to_document_store
+        end
+
+        def call
+          json_schema.expand_references!(store: document_store)
+          status, errors = json_schema.validate(response_body)
+          @message ||= errors.map(&:to_s).to_sentence
+          status
+        end
+
+        protected
+
+        attr_reader :document_store
+
+        def controller_path
+          response.request.filtered_parameters[:controller]
+        end
+
+        def action
+          response.request.filtered_parameters[:action]
+        end
+
+        def schema_directory
+          ActiveModelSerializers.config.schema_path
+        end
+
+        def schema_full_path
+          "#{schema_directory}/#{schema_path}"
+        end
+
+        def schema_path_default
+          "#{controller_path}/#{action}.json"
+        end
+
+        def schema_data
+          load_json_file(schema_full_path)
+        end
+
+        def response_body
+          load_json(response.body)
+        end
+
+        def json_schema
+          @json_schema ||= JsonSchema.parse!(schema_data)
+        end
+
+        def add_schema_to_document_store
+          Dir.glob("#{schema_directory}/**/*.json").each do |path|
+            schema_data = load_json_file(path)
+            extra_schema = JsonSchema.parse!(schema_data)
+            document_store.add_schema(extra_schema)
+          end
+        end
+
+        def load_json(json)
+          JSON.parse(json)
+        rescue JSON::ParserError => ex
+          raise InvalidSchemaError, ex.message
+        end
+
+        def load_json_file(path)
+          load_json(File.read(path))
+        rescue Errno::ENOENT
+          raise MissingSchema, "No Schema file at #{schema_full_path}"
+        end
+
+        def require_json_schema!
+          require 'json_schema'
+        rescue LoadError
+          raise LoadError, "You don't have json_schema installed in your application. Please add it to your Gemfile and run bundle install"
+        end
+      end
+    end
+  end
+end

test/active_model_serializers/test/schema_test.rb

@@ -0,0 +1,128 @@
+require 'test_helper'
+
+module ActiveModelSerializers
+  module Test
+    class SchemaTest < ActionController::TestCase
+      include ActiveModelSerializers::Test::Schema
+
+      class MyController < ActionController::Base
+        def index
+          render json: profile
+        end
+
+        def show
+          index
+        end
+
+        def name_as_a_integer
+          profile.name = 1
+          index
+        end
+
+        def render_using_json_api
+          render json: profile, adapter: :json_api
+        end
+
+        def invalid_json_body
+          render json: ''
+        end
+
+        private
+
+        def profile
+          @profile ||= Profile.new(name: 'Name 1', description: 'Description 1', comments: 'Comments 1')
+        end
+      end
+
+      tests MyController
+
+      def test_that_assert_with_a_valid_schema
+        get :index
+        assert_response_schema
+      end
+
+      def test_that_raises_a_minitest_error_with_a_invalid_schema
+        message = "#/name: failed schema #/properties/name: For 'properties/name', \"Name 1\" is not an integer. and #/description: failed schema #/properties/description: For 'properties/description', \"Description 1\" is not a boolean."
+
+        get :show
+
+        error = assert_raises Minitest::Assertion do
+          assert_response_schema
+        end
+        assert_equal(message, error.message)
+      end
+
+      def test_that_raises_error_with_a_custom_message_with_a_invalid_schema
+        message = 'oh boy the show is broken'
+
+        get :show
+
+        error = assert_raises Minitest::Assertion do
+          assert_response_schema(nil, message)
+        end
+        assert_equal(message, error.message)
+      end
+
+      def test_that_assert_with_a_custom_schema
+        get :show
+        assert_response_schema('custom/show.json')
+      end
+
+      def test_that_assert_with_a_hyper_schema
+        get :show
+        assert_response_schema('hyper_schema.json')
+      end
+
+      def test_simple_json_pointers
+        get :show
+        assert_response_schema('simple_json_pointers.json')
+      end
+
+      def test_simple_json_pointers_that_doesnt_match
+        get :name_as_a_integer
+
+        assert_raises Minitest::Assertion do
+          assert_response_schema('simple_json_pointers.json')
+        end
+      end
+
+      def test_json_api_schema
+        get :render_using_json_api
+        assert_response_schema('render_using_json_api.json')
+      end
+
+      def test_that_assert_with_a_custom_schema_directory
+        original_schema_path = ActiveModelSerializers.config.schema_path
+        ActiveModelSerializers.config.schema_path = 'test/support/custom_schemas'
+
+        get :index
+        assert_response_schema
+
+        ActiveModelSerializers.config.schema_path = original_schema_path
+      end
+
+      def test_with_a_non_existent_file
+        message = %r{.* - No Schema file at test/support/schemas/non-existent.json}
+
+        get :show
+
+        error = assert_raises ActiveModelSerializers::Test::Schema::MissingSchema do
+          assert_response_schema('non-existent.json')
+        end
+        assert_match(message, error.message)
+      end
+
+      def test_that_raises_with_a_invalid_json_body
+        message = 'A JSON text must at least contain two octets!'
+
+        get :invalid_json_body
+
+        error = assert_raises ActiveModelSerializers::Test::Schema::InvalidSchemaError do
+          assert_response_schema('custom/show.json')
+        end
+
+        assert_equal(message, error.message)
+      end
+    end
+  end
+end

test/support/custom_schemas/active_model_serializers/test/schema_test/my/index.json

@@ -0,0 +1,6 @@
+{
+  "properties": {
+    "name" : { "type" : "string" },
+    "description" : { "type" : "string" }
+  }
+}

test/support/schemas/active_model_serializers/test/schema_test/my/index.json

@@ -0,0 +1,6 @@
+{
+  "properties": {
+    "name" : { "type" : "string" },
+    "description" : { "type" : "string" }
+  }
+}

test/support/schemas/active_model_serializers/test/schema_test/my/show.json

@@ -0,0 +1,6 @@
+{
+  "properties": {
+    "name" : { "type" : "integer" },
+    "description" : { "type" : "boolean" }
+  }
+}

test/support/schemas/custom/show.json

@@ -0,0 +1,7 @@
+{
+  "id": "file://custom/show.json#",
+  "properties": {
+    "name" : { "type" : "string" },
+    "description" : { "type" : "string" }
+  }
+}

test/support/schemas/hyper_schema.json

@@ -0,0 +1,93 @@
+{
+  "$schema": "http://json-schema.org/draft-04/hyper-schema",
+  "title": "Profile",
+  "description": "Profile schema",
+  "stability": "prototype",
+  "strictProperties": true,
+  "type": [
+    "object"
+  ],
+  "definitions": {
+    "name": {
+      "description": "unique name of profile",
+      "readOnly": true,
+      "type": [
+        "string"
+      ]
+    },
+    "description": {
+      "description": "description of profile",
+      "readOnly": true,
+      "type": [
+        "string"
+      ]
+    },
+    "identity": {
+      "anyOf": [
+        {
+          "$ref": "/schemata/profile#/definitions/name"
+        }
+      ]
+    }
+  },
+  "links": [
+    {
+      "description": "Create a new profile.",
+      "href": "/profiles",
+      "method": "POST",
+      "rel": "create",
+      "schema": {
+        "properties": {
+        },
+        "type": [
+          "object"
+        ]
+      },
+      "title": "Create"
+    },
+    {
+      "description": "Delete an existing profile.",
+      "href": "/profiles/{(%2Fschemata%2Fprofile%23%2Fdefinitions%2Fidentity)}",
+      "method": "DELETE",
+      "rel": "destroy",
+      "title": "Delete"
+    },
+    {
+      "description": "Info for existing profile.",
+      "href": "/profiles/{(%2Fschemata%2Fprofile%23%2Fdefinitions%2Fidentity)}",
+      "method": "GET",
+      "rel": "self",
+      "title": "Info"
+    },
+    {
+      "description": "List existing profiles.",
+      "href": "/profiles",
+      "method": "GET",
+      "rel": "instances",
+      "title": "List"
+    },
+    {
+      "description": "Update an existing profile.",
+      "href": "/profiles/{(%2Fschemata%2Fprofile%23%2Fdefinitions%2Fidentity)}",
+      "method": "PATCH",
+      "rel": "update",
+      "schema": {
+        "properties": {
+        },
+        "type": [
+          "object"
+        ]
+      },
+      "title": "Update"
+    }
+  ],
+  "properties": {
+    "name": {
+      "$ref": "/schemata/profile#/definitions/name"
+    },
+    "description": {
+      "$ref": "/schemata/profile#/definitions/description"
+    }
+  },
+  "id": "/schemata/profile"
+}

test/support/schemas/render_using_json_api.json

@@ -0,0 +1,43 @@
+{
+  "$schema": "http://json-schema.org/draft-04/schema#",
+  "id": "",
+  "type": "object",
+  "properties": {
+    "data": {
+      "id": "/data",
+      "type": "object",
+      "properties": {
+        "id": {
+          "id": "/data/id",
+          "type": "string"
+        },
+        "type": {
+          "id": "/data/type",
+          "type": "string"
+        },
+        "attributes": {
+          "id": "/data/attributes",
+          "type": "object",
+          "properties": {
+            "name": {
+              "id": "/data/attributes/name",
+              "type": "string"
+            },
+            "description": {
+              "id": "/data/attributes/description",
+              "type": "string"
+            }
+          }
+        }
+      },
+      "required": [
+        "id",
+        "type",
+        "attributes"
+      ]
+    }
+  },
+  "required": [
+    "data"
+  ]
+}

test/support/schemas/simple_json_pointers.json

@@ -0,0 +1,10 @@
+{
+  "properties": {
+    "name": {
+      "$ref": "file://custom/show.json#/properties/name"
+    },
+    "description": {
+      "$ref": "file://custom/show.json#/properties/description"
+    }
+  }
+}