Schema Validation and Aggregation Patterns: Inside Our MongoDB Growth Track

Why a "schemaless" database still needs a schema

MongoDB's pitch is flexible, schema-free documents — but any collection that backs a real application eventually needs guardrails. A posts collection where comments can silently become a string instead of an array, or where creator is missing entirely, doesn't fail loudly at write time. It fails quietly, three services downstream, on a client's production system, when something tries to .map() over a field that isn't there.

As part of our database growth track, our engineers work through exactly that problem space: how far you can push MongoDB's document validation, how aggregation pipelines reshape data server-side instead of in application code, and what it actually costs to insert a million documents.

Enforcing structure with $jsonSchema

MongoDB's validator option on createCollection lets you define a JSON Schema that every write has to satisfy:

db.createCollection("posts", {
  validator: {
    $jsonSchema: {
      bsonType: "object",
      required: ["title", "text", "creator", "comments"],
      properties: {
        title: { bsonType: "string", description: "must be a string and is required" },
        text: { bsonType: "string", description: "must be a string and is required" },
        creator: { bsonType: "objectId", description: "must be an objectId and is required" },
        comments: {
          bsonType: "array",
          description: "must be an array and is required",
          items: {
            bsonType: "object",
            required: ["text", "author"],
            properties: {
              text: { bsonType: "string", description: "must be a string and is required" },
              author: { bsonType: "objectId", description: "must be an objectId and is required" },
            },
          },
        },
      },
    },
  },
});

The interesting part isn't the schema itself — it's collMod, which lets you change validation rules on a collection that already has data in it, without dropping and recreating anything:

db.runCommand({
  collMod: "posts",
  validator: { $jsonSchema: { /* ...same shape... */ } },
  validationAction: "warn",
});

That validationAction: "warn" flag is the practical detail we drill into every engineer who goes through this module. Flip it to "error" and MongoDB rejects any write that violates the schema outright. Leave it at "warn" and violations are logged but still allowed through. That gives you a migration path we use on real client collections: tighten the schema, run in "warn" mode against production traffic, see what actually breaks in the logs, fix the offending writers, then switch to "error". It's the same "shadow mode" pattern you'd use for a risky application-level rollout, just implemented at the storage layer.

Reshaping data with the aggregation pipeline

The second thread in this module is aggregation — pushing grouping, sorting, and string manipulation into the database instead of pulling raw documents into application code and transforming them there. Working against a sample dataset of ~1.6 million person records (10. persons.json), a few pipeline stages show what this buys you:

db.persons.aggregate([
  { $match: { gender: "female" } },
  { $group: { _id: { state: "$location.state" }, totalPersons: { $sum: 1 } } },
  { $sort: { totalPersons: -1 } },
]);

Filter, then group, then sort — three stages, each one operating on the output of the last, entirely on the server. No document leaves the database until the aggregation is done, which matters once "documents" means millions of rows instead of a handful.

The $project stage is where things get more interesting than a simple field selection. Building a properly capitalized full name out of first/last subfields, using only aggregation operators (no application code, no post-processing):

db.persons.aggregate([
  {
    $project: {
      _id: 0,
      gender: 1,
      fullName: {
        $concat: [
          { $toUpper: { $substrCP: ["$name.first", 0, 1] } },
          { $substrCP: ["$name.first", 1, { $subtract: [{ $strLenCP: "$name.first" }, 1] }] },
          " ",
          { $toUpper: { $substrCP: ["$name.last", 0, 1] } },
          { $substrCP: ["$name.last", 1, { $subtract: [{ $strLenCP: "$name.last" }, 1] }] },
        ],
      },
    },
  },
]);

That's a title-case transformation built entirely from $substrCP, $toUpper, $strLenCP, and $subtract — composable expression operators nested inside each other. It reads dense the first time, but it's the same logic you'd write in JavaScript (s[0].toUpperCase() + s.slice(1)), just expressed as a data pipeline instead of imperative code. The trade-off is legibility versus location: harder to read at a glance, but it runs where the data already lives, with no round trip.

What breaks at scale

Schema design and clean pipelines only matter if the database can actually take the write volume. 10. credit-rating.js is a deliberately blunt instrument we use for finding out:

conn = new Mongo();
db = conn.getDB("credit");

for (let i = 0; i < 1000000; i++) {
  db.ratings.insertOne({
    person_id: i + 1,
    score: Math.random() * 100,
    age: Math.floor(Math.random() * 70) + 18,
  });
}

A million sequential insertOne calls is intentionally the naive approach — no batching, no insertMany, no bulk write API. Running it is a fast way for an engineer to feel the difference between "MongoDB can handle a million documents" (true) and "issuing a million individual round-trips is the right way to get them there" (not true). It's the kind of script we have engineers write once to establish a baseline before reaching for bulkWrite or insertMany with ordered writes disabled — you want the slow version to hurt before you optimize it away.

Where this fits

None of these are production files — they're the kind of focused, disposable scripts an engineer writes while building the mental model for how a database actually behaves under constraints, rather than trusting the marketing description of it. A few concrete takeaways that carry over into real client project work:

The common thread — and what we want every engineer on the team to carry into client work — is treating the database as more than a dumb key-value store: validation rules, aggregation pipelines, and write patterns are all levers that belong to the data layer, not just the application layer, and it's worth knowing exactly what each one costs before reaching for it on a real system.