Filter collections of data

Many applications perform authorization over large collections of data that cannot be loaded into memory. Often index pages showing users a number of resources, like the repositories they can access, will need to use data filtering. The data filtering API provides support for these use cases seamlessly without requiring you to alter your policy.

Get all authorized resources

In Enforce authorization we discussed resource-level authorization. The authorize method tells you whether a specific resource is authorized. But to fetch all authorized resources, we need to use authorized_resources instead:

get "/repos" do
  repositories = oso.authorized_resources(
    get_current_user(),
    "read",
    DF::Repository)

  serialize(repositories)
end

To use this API, you must pass some additional information to register_class so that Oso knows how to retrieve your application’s objects.

Implementing data filtering query functions

To use data filtering, you tell Oso how to make queries to your data store for the resources used in your policy. Oso uses Query objects to query your data store. A Query represents a set of filters to apply to a collection of data.

You can use any type as a Query. Many ORMs have these built in, but you may have your own representation if your resources are retrieved from an external service, or with a lower-level database API.

You implement three functions to tell Oso how to work with your Query:

• build_query(filters) -> Query: Creates a query from a list of authorization filters produced by evaluating the Oso policy.
• exec_query(query) -> List[Object]: Executes the query, returning the list of objects retrieved by the query.
• combine_query(q1, q2) -> Query: Combines two queries q1 and q2 together such that the new query returns the UNION of q1 and q2 (all results from each).

  # This is an example implementation for the Sequel ORM, but you can
  # use any ORM with this API.
  class SequelAdapter
    def build_query(filter)
      types = filter.types
      query = filter.relations.reduce(filter.model) do |q, rel|
        rec = types[rel.left].fields[rel.name]
        q.join( rel.right.table_name,
          "#{rel.left.table_name}.#{rec.my_field}" =>
        "#{rel.right.table_name}.#{rec.other_field}"
        )
      end

      args = []
      sql = filter.conditions.map do |conjs|
        conjs.reduce('true') do |sql, conj|
          "(#{sql} AND #{sqlize(conj, args)})"
        end
      end.reduce('false') do |sql, clause|
        "(#{sql} OR #{clause})"
      end 

      query.where(Sequel.lit(sql, *args)).distinct
    end

    def execute_query(query)
      query.to_a
    end

    OPS = {
      'Eq' => '=', 'In' => 'IN', 'Nin' => 'NOT IN', 'Neq' => '!=',
      'Lt' => '<', 'Gt' => '>', 'Leq' => '<=', 'Geq' => '>='
    }.freeze

    private

    def sqlize(cond, args)
      lhs = add_side cond.left, args
      rhs = add_side cond.right, args
      "#{lhs} #{OPS[cond.cmp]} #{rhs}"
    end

    def add_side(side, args)
      if side.is_a? ::Oso::Polar::Data::Filter::Projection
        "#{side.source.table_name}.#{side.field || :name}"
      elsif side.is_a? DF::Repository
        args.push side.name
        '?'
      else
        args.push side
        '?'
      end
    end
  end

  OSO.register_class(User)
  OSO.register_class(
    Repository,
    name: "Repository",
    fields: {
      is_public: PolarBoolean
    },
  )

  OSO.data_filtering_adapter = SequelAdapter.new

  OSO.load_files(["main.polar"])

When you call authorized_resources, Oso will create a query using the build_query function with filters obtained by running the policy. For example, in Write Polar Rules we wrote the rule:

has_permission(_user: User, "read", repository: Repository) if
	repository.is_public = true;

This rule would produce the filters: [Filter(kind=Eq, field="is_public", value=true)]. Oso then uses SQLAlchemy in our example to create a query and retrieve repositories that have the is_public field as true from the database by calling the exec_query function. This pushes down filters to the database, allowing you to retrieve only authorized objects. Notably, the same rule can be executed using authorize and authorized_resources.

Adding filters on top of authorization

Often, you may want to add to the query after it is authorized. Let’s say we want to order Repositories by name.

To do this, we can use the authorized_query API:

get "/repos" do
  query = oso.authorized_query(
    get_current_user(),
    "read",
    Repository)

  # Use the ORM's Query API to alter the query before it is
  # executed by the database with .all.
  repositories = query.order_by(:name).all

  serialize(repositories)
end

authorized_query returns the query object used by our ORM with authorization filters applied so that we can add additional filters, pagination, or ordering to it.

What’s next

In this brief example we covered what the data filtering API does. For a more detailed how to of using data filtering and implementing query builder functions, see the Data Filtering guide.

This is the end of Add to your app! For more detail on using Oso, see the guides section.

Connect with us on Slack

If you have any questions, or just want to talk something through, jump into Slack. An Oso engineer or one of the thousands of developers in the growing community will be happy to help.