Advanced Arel: When ActiveRecord Just Isn't Enough

ADVANCED AREL
WHEN ACTIVERECORD JUST ISN’T ENOUGH
Author.where(
name: "Cameron Dutro",
username: "@camertron"
)

`WHOAMI`
CAMERON DUTRO
INTERNATIONAL MAN OF MYSTERY
US GOVERNMENT
@CAMERTRON
GITHUB.COM/CAMERTRON

`WHOAMI`
CAMERON DUTRO
INTERNATIONAL ENGINEERING
TWITTER, INC
@CAMERTRON
GITHUB.COM/CAMERTRON

RAISE YOUR HAND IF...
Posts.find(:all,
joins: [
"JOIN comments ON comments.post_id = posts.id",
"JOIN authors ON authors.id = comments.author_id"
],
conditions: [
"authors.name = ? AND posts.active = ?",
"Barack Obama", true
]
)
2

RAISE YOUR HAND IF...
Posts
.joins(
"JOIN comments ON comments.post_id = posts.id",
"JOIN authors ON authors.id = comments.author_id")
.where(
)
3/4

HMM, LET’S SIMPLIFY?
Posts
.joins(:comments)
.joins(
)
.where(
)
3/4

HMM, LET’S SIMPLIFY?
Posts
.joins(:comments)
.joins(:comments => :author)
.where(
)
3/4

PROBLEMS
3/4
Posts
.joins(:comments)
.joins(
)
.where(
)

PROBLEMS
.joins(
)

PROBLEMS
.joins(
)
HAVE TO WRITE “JOIN” AND “ON”

PROBLEMS
.joins(
)
HAVE TO KNOW MYSQL SYNTAX

PROBLEMS
.joins(
)
NO SYNTAX CHECKING

PROBLEMS
.where(
)

PROBLEMS
.where(
)
CONFUSING TO MATCH ARGUMENTS WITH
QUESTION MARKS

PROBLEMS
.where(
)
QUESTION MARKS
NOT OBJECT-ORIENTED

PROBLEMS
.where(
)
QUESTION MARKS
NOT OBJECT-ORIENTED
NO SYNTAX CHECKING

STACKOVERFLOW
RAILSCASTS #202
BLOGS

STACKOVERFLOW
RAILSCASTS #202
BLOGS
COWORKERS

STACKOVERFLOW
RAILSCASTS #202
BLOGS
COWORKERS
FRIENDS

STACKOVERFLOW
RAILSCASTS #202
BLOGS
COWORKERS
FRIENDS
FAMILY MEMBERS

STACKOVERFLOW
RAILSCASTS #202
BLOGS
COWORKERS
FRIENDS
FAMILY MEMBERS
YOUR CAT

KEEP CALM
AND
AVOID
LITERAL
STRINGS IN
YOUR
QUERIES

A BETTER WAY
Post
.joins(:comments)
.joins(Comment.joins(:author).join_sources)
.where(
Author[:name].eq("Barack Obama")
.and(Post[:active].eq(true))
)

A BETTER WAY
Post
.joins(:comments)
.where(
)
DON’T HAVE TO KNOW SQL SYNTAX

A BETTER WAY
Post
.joins(:comments)
.where(
)
RUBY SYNTAX CHECKING

A BETTER WAY
Post
.joins(:comments)
.where(
)
OBJECT-ORIENTED (CHAINABLE)

A BETTER WAY
Post
.joins(:comments)
.where(
)
NO QUESTION MARKS

A BETTER WAY
Post
.joins(:comments)
.where(
)
NO QUESTION MARKS
EASY TO READ - IT’S JUST RUBY!

WHAT WE’LL COVER
ACTIVERECORD VS AREL
TABLES, COLUMNS
TERMINAL METHODS
SELECT, WHERE, JOIN, JOIN ASSOCIATION, ORDER
AND, OR , GREATER/LESS THAN, NOT EQUALS, ETC
MATCH, IN

ACTIVERECORD
DATABASE ABSTRACTION
NO NEED TO SPEAK A DIALECT OF SQL
PERSISTENCE
DATABASE ROWS AS RUBY OBJECTS
DOMAIN LOGIC
MODELS CONTAIN APPLICATION LOGIC, VALIDATIONS, ETC
MODELS DEFINE ASSOCIATIONS

AREL
“RELATIONAL ALGEBRA” FOR RUBY
BUILDS SQL QUERIES, GENERATES ASTS
APPLIES QUERY OPTIMIZATIONS
ENABLES CHAINING
“VEXINGLY UNDOCUMENTED”

AREL
KNOWS NOTHING ABOUT YOUR MODELS
KNOWS VERY LITTLE ABOUT YOUR DATABASE
DOES NOT RETRIEVE OR STORE DATA

AREL
KNOWS NOTHING ABOUT YOUR MODELS
KNOWS VERY LITTLE ABOUT YOUR DATABASE
DOES NOT RETRIEVE OR STORE DATA
ACTIVERECORD’S RESPONSIBILITY

ACTIVERECORD
DOES EVERYTHING ELSE

HIERARCHY
activerecord
arel
database

activerecord
arel
database
CONSTRUCT QUERY
EXECUTE QUERY
HIERARCHY

WHAT’S AN AST?
7
4 9
1 5
LEFT CHILD

WHAT’S AN AST?
7
4 9
1 5
LEFT CHILD RIGHT CHILD

WHAT’S AN AST?
5 * (6 + 3)
*
5 +
6 3

WHAT’S AN AST?
SELECT id, text FROM posts
SELECT
id text
<query>
FROM
posts

AREL-HELPERS GEM
gem install arel-helpers
gem “arel-helpers”, “~> 1.1.0”

TABLES AND COLUMNS
class Post < ActiveRecord::Base
has_many :comments
end

TABLES AND COLUMNS
has_many :comments
end
Post.arel_table[:id]

TABLES AND COLUMNS
has_many :comments
end
Post.arel_table[:text]

TABLES AND COLUMNS
has_many :comments
end
=> #<struct Arel::Attributes::Attribute ... >

TABLES AND COLUMNS
include ArelHelpers::ArelTable
has_many :comments
end

TABLES AND COLUMNS
include ArelHelpers::ArelTable
has_many :comments
end
Post[:id]
Post[:text]

RELATIONS
POP QUIZ! WHAT DOES THIS STATEMENT RETURN?
Post.select(:title)

RELATIONS
Post.select(:title)
A. [“Rails is Cool” ... ]

RELATIONS
Post.select(:title)
B. [#<Post title=”Rails is Cool”>, ... ]

RELATIONS
Post.select(:title)
C. <ActiveRecord::Relation ... >
B. [#<Post title=”Rails is Cool”>, ... ]

RELATIONS
query = Post.select(:title)

RELATIONS
query = query.select(:id)

RELATIONS
query.to_sql

RELATIONS
query.to_sql
=> SELECT title, id FROM `posts`

RELATIONS
query.to_sql
=> SELECT title, id FROM `posts`
RELATIONS CAN BE CHAINED!

THE SERENDIPITY OF “SELECT”

SELECT
Post.select([:id, :text]).to_sql
=> SELECT id, text FROM `posts`

SELECT
Post.select(:id).count.to_sql
=> NoMethodError: undefined method `to_sql' for 26:Fixnum

SELECT
WHAT HAPPENED??

SELECT
WHAT HAPPENED??
.count IS A TERMINAL METHOD

SELECT
Post.select([Post[:id].count, :text]).to_sql
=> SELECT COUNT(`posts`.`id`), text FROM `posts`

TERMINAL METHODS
EXECUTE IMMEDIATELY
DO NOT RETURN AN ActiveRecord::Relation
count
first, last
to_a
pluck
each, map, ETC

TERMINAL METHODS
Post.where(title: "Arel is Cool").each do |post|
puts post.text
end

TERMINAL METHODS
puts post.text
end
Post.where(title: "Arel is Cool").each_slice(3)

TERMINAL METHODS
puts post.text
end
Post.where(title: "Arel is Cool").each_slice(3)
BOTH EXECUTE THE QUERY IMMEDIATELY

SELECT
Post.select(Post[:visitors].sum).to_sql
=> SELECT SUM(`posts`.`visitors`) AS sum_id FROM `posts`

SELECT
Post.select(Post[:visitors].sum.as("visitor_total")).to_sql
=> SELECT SUM(`posts`.`views`) AS visitor_total FROM `posts`

SELECT
Post.select(Post[:visitors].maximum).to_sql
=> SELECT MAX(`posts`.`visitors`) AS max_id FROM `posts`

SELECT
Post.select(Post[:visitors].maximum).to_sql
=> SELECT MAX(`posts`.`visitors`) AS max_id FROM `posts`
Post.select(Post[:visitors].minimum).to_sql
=> SELECT MIN(`posts`.`visitors`) AS min_id FROM `posts`

SELECT
Post.select(
Arel::Nodes::NamedFunction.new(
"LENGTH", [Post[:text]]
).as("length")
).to_sql
=> SELECT LENGTH(`posts`.`text`) AS length FROM `posts`

SELECT
Post.select(
Arel::Nodes::NamedFunction.new(
).as("length")
).to_sql
include Arel::Nodes

SELECT
Post.select(
NamedFunction.new(
).as("length")
).to_sql
include Arel::Nodes

SELECT
Post.select(Arel.star).to_sql
=> SELECT * FROM `posts`

SELECT FROM
Post.select(:id).from(Post.select([:id, :text]).ast).to_sql
=> SELECT id FROM SELECT id, text FROM `posts`

WHERE
Post.where(title: "Arel is Cool").to_sql
=> SELECT ùsers`.* FROM ùsers`
WHERE ùsers`.`title` = 'Arel is Cool'
WITH ACTIVERECORD SUGAR

WHERE
Post.where(title: "Arel is Cool").to_sql
WITH ACTIVERECORD SUGAR
Post.where(Post[:title].eq("Arel is Cool")).to_sql
WITH AREL

WHERE
Post.where(Post[:title].not_eq("Arel is Cool")).to_sql
=> SELECT `posts`.* FROM `posts`
WHERE (`posts`.`title` != 'Arel is Cool')

WHERE
Post.where(Post[:title].not_eq("Arel is Cool")).to_sql
WHERE (`posts`.`title` != 'Arel is Cool')
Post.where(Post[:title].not_eq(nil)).to_sql
WHERE (`posts`.`title` IS NOT NULL)

WHERE
Post.where(Post[:visitors].gt(250)).to_sql
WHERE (`posts`.`visitors` > 250)

WHERE
Post.where(Post[:visitors].gt(250)).to_sql
WHERE (`posts`.`visitors` > 250)
Post.where(Post[:visitors].lt(250)).to_sql
WHERE (`posts`.`visitors` < 250)

WHERE
Post.where(Post[:visitors].gteq(250)).to_sql
WHERE (`posts`.`visitors` >= 250)
Post.where(Post[:visitors].lteq(250)).to_sql
WHERE (`posts`.`visitors` <= 250)

WHERE
Post.where(
Post[:title].eq("Arel is Cool").and(
Post[:id].eq(22).or(
Post[:id].eq(23)
)
)
).to_sql
WHERE (
`posts`.`title` = 'Arel is Cool' AND
(`posts`.`id` = 22 OR `posts`.`id` = 23)
)

WHERE
Post.where(
Post[:id].in(22, 23)
)
).to_sql
WHERE (
`posts`.`id` IN (22, 23)
)

WHERE
Post.where(
NamedFunction.new("LENGTH", [Post[:slug]]).gt(10)
)
).to_sql
WHERE (
LENGTH(`posts`.`slug`) > 10
)

JOIN
has_many :comments
end
class Comment < ActiveRecord::Base
belongs_to :post
has_one :author
end
class Author < ActiveRecord::Base
belongs_to :comment
end

JOIN
Author.joins(:comment).where(id: 42).to_sql
=> SELECT àuthors`.* FROM àuthors`
INNER JOIN `comments`
ON `comments`.ìd` = àuthors`.`comment_id`
WHERE àuthors`.ìd` = 42

JOIN
Author
.joins(:comment, :comment => :post)
.where(Post[:id].eq(42))
.to_sql
INNER JOIN `posts`
ON `posts`.`id` = `comments`.`post_id`
WHERE `posts`.`id` = 42

JOIN
Author
.joins(:comment)
.joins(Comment.joins(:post).join_sources)
.to_sql
INNER JOIN `posts`

JOIN
Author
.joins(Author.joins(:comment).join_sources)
.joins(Comment.joins(:post).join_sources)
.to_sql
INNER JOIN `posts`

JOIN
Author
.joins(
Author.arel_table.join(Comment.arel_table)
.on(Comment[:id].eq(Author[:comment_id]))
.join_sources
)
.joins(
Comment.arel_table.join(Post.arel_table)
.on(Post[:id].eq(Comment[:post_id]))
.join_sources
)
.to_sql

JOIN
Author
.joins(
Author.arel_table.join(Comment.arel_table, Arel::OuterJoin)
.join_sources
)
.joins(
Comment.arel_table.join(Post.arel_table, Arel::OuterJoin)
.join_sources
)
.to_sql

JOIN
LEFT OUTER JOIN `comments`
LEFT OUTER JOIN `posts`

JOIN
Author
.joins(
Author.arel_table.join(Comment.arel_table, Arel::OuterJoin)
.join_sources
)
.joins(
.join_sources
)
.to_sql
include ArelHelpers::JoinAssociation

JOIN
Author
.joins(join_association(Author, :comment, Arel::OuterJoin))
.joins(
.join_sources
)
.to_sql

JOIN
Author
.joins(join_association(Author, :comment, Arel::OuterJoin))
.joins(join_association(Comment, :post, Arel::OuterJoin))
.to_sql

JOIN
Author
.joins(
join_association(Author, :comment) do |assoc_name, join_conds|
join_conds.and(Comment[:created_at].lteq(Date.yesterday))
end
)
.joins(join_association(Comment, :post, Arel::OuterJoin))
.to_sql

JOIN
AND `comments`.`created_at` <= '2014-04-15'
LEFT OUTER JOIN `posts`

JOIN TABLES
class Course < ActiveRecord::Base
has_and_belongs_to_many :teachers
end
class Teacher < ActiveRecord::Base
has_and_belongs_to_many :courses
end

JOIN TABLES
class Course < ActiveRecord::Base
has_and_belongs_to_many :teachers
end
class Teacher < ActiveRecord::Base
has_and_belongs_to_many :courses
end
courses
teachers
courses_teachers

JOIN TABLES
Course.arel_table
course TABLE

JOIN TABLES
Course.arel_table
course TABLE
teacher TABLE

JOIN TABLES
Course.arel_table
Teacher.arel_table
course TABLE
teacher TABLE

JOIN TABLES
Course.arel_table
Teacher.arel_table
course TABLE
teacher TABLE
courses_teachers TABLE

JOIN TABLES
Course.arel_table
Teacher.arel_table
course TABLE
teacher TABLE
???? (no model class)
courses_teachers TABLE

JOIN TABLES
ct = Arel::Table.new(:courses_teachers)

JOIN TABLES
ct = Arel::Table.new(:courses_teachers)
Course
.joins(
Course.arel_table.join(Teacher.arel_table)
.on(Course[:id].eq(ct[:course_id]))
.and(Teacher[:id].eq(ct[:teacher_id]))
.join_sources
)
.to_sql

ORDER
Post.order(:visitors).to_sql
=> SELECT `posts`.* FROM `posts` ORDER BY visitors

ORDER
Post.order(:views).reverse_order.to_sql
=> SELECT `posts`.* FROM `posts` ORDER BY visitors DESC

ORDER
Post.order(:views).reverse_order.to_sql
Post.order(Post[:views].desc).to_sql

IN
Post.where(
Post.arel_table[:title].in(
Post.select(:title).where(id: 5).ast
)
).to_sql
=> SELECT `phrases`.* FROM `phrases`
WHERE `phrases`.`title` IN (
SELECT title FROM `phrases` WHERE `phrases`.`id` = 5
)

LIKE QUERIES WITH “MATCHES”

MATCHES
Post.where(Post[:title].matches("%arel%")).to_sql
=> SELECT `phrases`.* FROM `phrases`
WHERE (`phrases`.`key` LIKE x'256172656c25')

class QueryBuilder
extend Forwardable
attr_reader :query
def_delegators :@query, :to_a, :to_sql, :each
def initialize(query)
@query = query
end
protected
def reflect(query)
self.class.new(query)
end
end

class PostQueryBuilder < QueryBuilder
def initialize(query = nil)
super(query || post.unscoped)
end
def with_title_matching(title)
reflect(
query.where(post[:title].matches("%#{title}%"))
)
end
def with_comments_by(usernames)
reflect(
query
.where(Author[:username].in(usernames))
)
end
def since_yesterday
reflect(
query.where(post[:created_at].gteq(Date.yesterday))
)
end
private
def author
Author
end
def post
Post
end
end

def with_title_matching(title)
reflect(
query.where(post[:title].matches("%#{title}%"))
)
end

def with_comments_by(usernames)
reflect(
query
.where(Author[:username].in(usernames))
)
end

def since_yesterday
reflect(
query.where(
post[:created_at].gteq(Date.yesterday)
)
)
end

EXAMPLES
PostQueryBuilder.new
.with_comments_by(['camertron', 'catwithtail'])
.to_sql
ON `comments`.`post_id` = `posts`.ìd`
INNER JOIN àuthors`
ON àuthors`.`comment_id` = `comments`.ìd`
WHERE àuthors`.ùsername` IN (
'camertron', 'catwithtail'
)

EXAMPLES
.with_title_matching("arel").to_sql
) AND (`posts`.`title` LIKE '%arel%')

EXAMPLES
.with_title_matching(`arel`).since_yesterday.to_sql
) AND (`posts`.`title` LIKE '%arel%')
AND (`posts`.`created_at` >= '2014-04-20')

Sigh... this is all so complicated.
Surely there’s a tool out there...“
”

SCUTTLE
sql-parser (java)
http://github.com/camertron/sql-parser
scuttle-rb (jRuby gem)
http://github.com/camertron/scuttle-rb
scuttle-server (sinatra jRuby)
http://github.com/camertron/scuttle-server

Advanced Arel: When ActiveRecord Just Isn't Enough

More Related Content

Viewers also liked (14)

Similar to Advanced Arel: When ActiveRecord Just Isn't Enough (17)

Recently uploaded (20)

Advanced Arel: When ActiveRecord Just Isn't Enough