DEV DAY

Advanced Querying Techniques With

Aggregation Pipelines
Hands-On Session

● No stupid questions
● Actively discuss
● Be respectful
Agenda
● MongoDB Query API
● Aggregation Pipeline Framework
● Filtering data
● Grouping data
● Modifying data
MongoDB Query API
MongoDB Query API: rich MongoDB
and expressive {
customer_id : 1,
first_name : "Mark",
last_name : "Smith",
● Find anyone with phone # “1-212…”
Expressive
● Check if the person with number “555…” is on the “do not city : "San Francisco",
queries call” list
phones: [
● Find the best offer for the customer at geo coordinates of {
Geospatial 42nd St. and 6th Ave number : "1-212-777-1212",
type : "work"
Text search ● Find all tweets that mention the firm within the last 2 days
},
● Count and sort number of customers by city, compute {
Aggregation min, max, and average spend
number : "1-212-777-1213",
● Add an additional phone number to Mark Smith’s record type : "cell"
Native binary without rewriting the document }
JSON support ● Update just 2 phone numbers out of 10
● Sort on the modified date ]
}
JOIN ● Query for all San Francisco residences, lookup their
($lookup) transactions, and sum the amount by person

Graph queries ● Query for all people within 3 degrees of separation

($graphLookup) from Mark
Queries in Node.js
const db = conn.db("library");
const collection = db.collection("books");
Queries in Node.js
const document = await collection.findOne({
_id: "0449005615"
});
const documents = await collection.find({
authors.name: "Laura Hillenbrand"
}).toArray();
Queries in Node.js
const result = await collection.insertOne({title: "Lord of the Ring"});
const result = await collection.insertMany(arrayOfBooks);

const result = await collection.updateOne({_id: "0449005615"}, {$set: {title:

"The Great Query"}});
const result = await collection.updateMany({authors.name: "Laura Hillenbrand"},
{$set: {authors.name: "Laura Hasanewname"}});

const result = await collection.deleteOne({_id: "0449005615"});

const result = await collection.deleteMany({authors.name: "Laura Hillenbrand"});
Aggregation
Pipelines
Aggregations
Advanced data processing pipeline
for transformations
and analytics
Multiple stages

Similar to a unix pipe

• Construct modular, composable processing pipelines

Rich Expressions

Example Aggregation Command

on the Orders Collection:

db.orders.aggregate( [
$match stage {$match: { status: "A" } },
$group stage {$group: { _id: "$cust_id", total: { $sum: "$amount" } } }
] )
Aggregation features
A feature rich framework for data transformation and Analytics

Pipeline Stages Operators

• $match • $lookup • Mathematical • Conditionals • String
• $group • $merge $add, $abs, $subtract,
$multiply, $divide, $log,
$and, $or, $eq, $lt, $lte, $gt,
$gte, $cmp, $cond, $switch,
$toUpper, $toLower, $substr, $strcasecmp,
$concat, $split, etc.
$log10, $stdDevPop, $in, etc.
• $facet • $project $stdDevSam, $avg, • Literals
$sqrt, $pow, $sum, $zip, • Temporal
• $geoNear • $search $convert, $round, etc. $exp, $let, $literal, $map, $type, etc.
Window Functions
• $graphLookup • $sort • Array $dateAdd, $dateDiff,
• Regex
$push, $reduce, $dateSubtract, $dateTrunc $regexFind, $regexMatch, etc
• $setWindowFields $reverseArray,
$dateFromParts,
$addToSet,
$dateToParts, • Trigonometry
• $unionWith $arrayElemAt, $slice,
$dateFromString,
etc. $sin, $cos, $degreesToRadians, etc.
$dateToString,
• $unwind $dayOfMonth, $isoWeek,
$minute, $month, $year, etc. • Custom Aggregation
• ...and more Expressions

PRACTICAL MONGODB AGGREGATIONS EBOOK

Compared to SQL JOINs and aggregation
SELECT
city,
SUM(annual_spend) Total_Spend,
SQL queries have a
AVG(annual_spend) Average_Spend,
nested structure
MAX(annual_spend) Max_Spend,
COUNT(annual_spend) customers
FROM (
SELECT t1.city, customer.annual_spend
Understanding the
FROM customer
outer layers requires
LEFT JOIN (
understanding the
SELECT address.address_id, city.city,
inner ones
address.customer_id, address.location
FROM address LEFT JOIN city
ON address.city_id = city.city_id
) AS t1
ON
(customer.customer_id = t1.customer_id AND
t1.location = "home")
) AS t2
So SQL has to be read
GROUP BY city; “inside-out”
Complex queries fast to
create, optimize, and maintain
db.customers.aggregate([
{
$unwind: "$address", These “phases” are distinct and
}, easy to understand
{
$match: {"address.location": "home"}
},
{
$group: {
_id: "$address.city",
totalSpend: {$sum: "$annualSpend"},
averageSpend: {$avg: "$annualSpend"}, They can be thought about
maximumSpend: {$max: "$annualSpend"}, in order… no nesting
customers: {$sum: 1}
}
}
])
MongoDB’s aggregation framework has the flexibility you need to get
value from your data, but without the complexity and fragility of SQL
 Build and test sample aggregation
pipelines with the easy-to-use Aggregation Pipeline
Builder

Choose from dozens of MongoDB aggregation stages

and over 150 operators

Drag-and-drop stages to test and apply

changes quickly

See sample document results to verify

transformations and aggregations at each stage

Aggregation Pipeline Builder Export your finished pipeline code to your language
of choice to execute in application code
MongoDB Compass
Aggregation
Stages
 ● Filters the data, similar to .find()
● Can use various operators such
as $gt, $lt, $and, $or

db.books.aggregate([
{
$match: {
pages: 100
}

$match ]);
}
 ● Returns a subset of the fields

db.books.aggregate([
{
$project: {title:1, cover: 1}

$project ]);
}
 ● Limits the number of returned
documents

db.books.aggregate([
{
$limit: 5

$limit ]);
}
 ● Match the exact array
db.collection.find({
tags: ['red', 'blank']
});

● Query for an array containing

items
db.collection.find({
tags: { $all: ['red', 'blank'] }
});

Querying ● Query for an array that contains

an element
db.collection.find({

arrays tags: 'red'

});
At the end of this exercise, you
will be able to:
● Open the Mongosh shell in Compass

● Run simple queries using $match, $project, and $limit

● Combine aggregation stages together

● Query data based on values inside arrays

 ● Counts the number of documents
in the pipeline

db.books.aggregate([
{
$count: "fieldName"

$count ]);
}
 ● Group documents together

db.books.aggregate([
{$group:{
_id: "$year",
totalPages: {$sum: "$pages"}
}}

$group ])
 ● Joins documents across
collections
● Because you can do joins doesn’t
meet you should use them

db.authors.aggregate([
{$lookup: {
from: "books",
localField: "books",
foreignField: "_id",
as: "booksWritten"
}

$lookup ])
At the end of this exercise, you
will be able to:
● Run aggregation pipelines with the $count stage

● Run aggregation pipelines that $group data

● Create joins using the $lookup stage

 ● Modifies the documents in the
pipeline

db.books.aggregate([
{$set: {
readingTimeHours:
{$divide: [{$multiply: ["$pages", 2]},
60]}}},

$set ])
 ● Exports the data into another
database and/or collection

{ $out: {
db: "<output-db>",
coll: "<output-collection>"
}}

$out
At the end of this exercise, you
will be able to:
● Run an aggregation pipeline that modifies the data

● Export the modified data into a new collection

Thank you!

https://mdb.link/developer-day-toronto