How it works
A table is an ordered map of rows in lexicographic row-key order. Each row contains column families, qualifiers within those families, and cell versions with timestamps. You define families in advance. You can create qualifiers as needed, which supports sparse records.
Bigtable divides consecutive row-key ranges into tablets and assigns tablets to nodes that serve requests. It splits and moves tablets as data and traffic change. This uses range partitioning, not consistent hashing, so nearby keys stay together. The first bytes of your row key strongly affect load distribution.
The storage engine uses an LSM-style write path. Writes enter a durable log and an in-memory structure, then move into immutable SSTables. Reads combine relevant data from memory and disk. Background compaction merges files and removes old versions that qualify for removal. Bloom filters help prevent unnecessary file reads.
Bigtable separates compute and storage. Google's distributed storage system, Colossus, holds durable files without dependence on a node's local disk. Bigtable can restore service through tablet reassignment without a full dataset copy to a replacement node. However, failures and load redistribution can still affect latency.
An instance can contain multiple clusters with asynchronous replication between them. Application profiles control request routes and access to single-row transactional operations. Single-cluster access provides strong consistency. Requests across replicated clusters generally introduce eventual consistency and require an explicit failover policy.
What it gives you in a design
- Ordered key and range access
- Core access patterns include point reads, prefix scans, and bounded row-key scans. Design keys for the queries first. A filter on a large scan does not make it an efficient indexed lookup.
- Sparse, versioned cells
- Rows can have different qualifiers, and cells can keep multiple versions with timestamps. This suits sparse attributes and time-based data, but you must deliberately choose which versions to keep.
- Single-row atomic operations
- Mutations within one row are atomic. Conditional mutations support compare-and-set-style updates. Bigtable has no general transaction across multiple rows or tables. Transactional operations require a compatible application profile.
- Column-family garbage collection
- Retention policies can remove versions by age, version count, or combinations of these rules. Collection is asynchronous. Use read filters when an application must immediately exclude expired versions.
- Managed scaling and replication
- Bigtable manages tablet placement and supports autoscaling for compatible cluster configurations. Replicated clusters provide regional placement and failover options. However, more nodes or replicas do not fix a single hot row.
- Change streams
- Change streams expose data changes for downstream processes, replication pipelines, and materialized views. Consumers still need checkpoints, retry control, and idempotent processing. They must not assume that external effects occur exactly once.
Scaling, replication and failure
Spread writes across key ranges that can become independent tablets. A timestamp-first key concentrates new writes at one end of the table. An entity-first key, such as device_id#time_bucket#timestamp, can distribute traffic when many devices are active. A very busy entity may still need a shard prefix. Reads must then access multiple shards.
Tablets split and move automatically, but Bigtable does not split a row across nodes. More nodes help only when enough separate key ranges have high traffic. Plan capacity with representative reads, writes, row sizes, and scans. Keep spare capacity for failures and load redistribution. Do not state a universal throughput number per node.
One cluster provides strongly consistent responses. Replication between clusters is asynchronous. Requests across clusters can return stale data, even after a write succeeds elsewhere. Send related reads and writes to the same cluster when read-your-writes matters. Explain how failover can temporarily weaken that guarantee.
Bigtable resolves concurrent updates to the same cell in different clusters through timestamp-based conflict resolution, not application-level merges. Avoid independent regional writes for inventory, balances, or other invariants that require coordination. Retry temporary errors with backoff. Distinguish retry-safe mutations from operations such as increments, whose results can be uncertain after a timeout.
Limits and defaults to know
Many of these values are configurable. Check the current documentation before you use an exact value.
| Row-key size | The maximum row-key length is 4 KiB. Keep keys much smaller when possible because they increase storage and process costs. |
|---|---|
| Cell value size | The maximum cell value size is 100 MiB. Google's schema guidance recommends values below 10 MiB. |
| Row size | The maximum row size is 256 MiB. Google's schema guidance recommends rows below 100 MiB. |
| Transaction scope | Atomicity applies to one row, not an entire batch of rows. Single-row transactional operations require a compatible application profile. |
| Change-stream retention | You can set change-stream retention from 1 to 7 days. Consumers that lag beyond this period need another recovery source. |
Use it when
- The workload has many writes and predictable point reads or bounded scans across a very large dataset.
- You can explain a row-key design that distributes load and keeps related data close for important queries.
- Telemetry, event history, or precomputed aggregates need a database that serves requests, not a warehouse query engine.
- Sparse records and per-cell versions are useful. One row or upstream processes can ensure correct results.
- Managed regional replication is valuable. The design can state which operations tolerate stale reads.
Avoid it when
- Correct results depend on atomic transactions across accounts, seats, orders, or other records with independent keys.
- The main workload uses ad hoc joins, large analytical aggregations, or searches across arbitrary fields.
- Most traffic targets one row that cannot split. The application cannot shard that state or use a cache to handle reads.
- The dataset is small enough that a relational database would provide simpler queries, constraints, and operations.
What interviewers ask about it
When Bigtable is on your whiteboard, expect these follow-up questions. Each answer gives the core of a strong answer.
- How would you design a row key for device telemetry?
- Start with the reads: one device's recent measurements suggest device_id#time_bucket#timestamp. Check whether a few devices dominate writes. Add a shard component only if necessary. It changes one range scan into multiple scans that the application must merge.
- Why do more nodes not solve every hotspot?
- Bigtable distributes tablets, but it cannot split one row across nodes. One hot row needs an application change, such as sharded counters, time buckets, caching, or less frequent updates.
- Can a client read its own write after regional failover?
- Not necessarily. The destination cluster may not have received that write yet. Single-cluster routing provides strong consistency during normal operation. But the failover policy must explicitly address replication lag and acceptable stale reads.
- How do you make writes safe to retry?
- Prefer deterministic row keys and repeatable cell assignments. Use explicit timestamps when version identity matters. Increments and other non-idempotent operations need extra care. A timeout can occur after the server commits the operation.
- Can Bigtable enforce a uniqueness constraint across rows?
- Bigtable has no general cross-row uniqueness constraint or multi-row transaction. You can sometimes place the contested claim in one row and use a conditional mutation. But dependent updates to other rows require a separate consistency and recovery plan.
- Does a retention policy immediately hide expired data?
- No. Garbage collection runs asynchronously and does not provide an exact deletion deadline. Apply timestamp or version filters at read time for correct query results. Treat the time of physical deletion as a separate operational concern.
Common mistakes
- You put a timestamp first in the row key for a workload with many appends. This concentrates writes on the newest key range.
- You call Bigtable an analytical columnar database because it uses column families. Its main access model uses keyed rows and ranges.
- You assume replication provides global strong consistency or that regional failover automatically preserves read-your-writes.
- You treat multi-row write batches as transactions, or you retry increments as if they were idempotent assignments.
Alternatives
- Apache Cassandra
- It suits similar wide-column workloads when partition-key distribution, deployment portability, or Cassandra's configurable consistency matters more than Bigtable's managed range-partitioned service.
- Amazon DynamoDB
- It suits AWS-native key-value workloads when partition-key access, secondary indexes, and bounded transactional operations meet requirements better than Bigtable's ordered row ranges.
- Google Spanner
- It is better when correct results depend on relational queries, secondary indexes, and strongly consistent transactions across rows.
- Google BigQuery
- It is better for warehouse analytics, broad scans, joins, and ad hoc aggregation than for keyed data access in a request path.
Where it is used
Bigtable is a good fit for these interview questions. Each line tells you why.
- Design a metrics monitoring system like DatadogBigtable can store metrics in time buckets and rollups when row keys control cardinality, distribute data input, and limit dashboard reads.
- Design Google AnalyticsIt can serve precomputed aggregates for tenants and time windows. Stream processing handles deduplication and late events.
- Design a distributed tracing systemBigtable suits span storage with trace IDs as keys. Trace searches by attribute require a separate index or a view that serves requests.
- Design an ad click aggregatorIt can hold sharded window aggregates if the design makes replay safe. The design must not treat stored counters as a transactional billing ledger.
- Design a distributed key-value storeBigtable provides a specific example of range partitioning, LSM-based storage, single-row atomicity, and asynchronous replication across clusters.
Common questions
What is Google Bigtable used for?
Google Bigtable stores large, sparse datasets that need high-volume writes and efficient key-based reads. Common uses include telemetry, time-series history, event data, and precomputed views for application reads. The key factor is whether the important queries map directly to row keys and bounded ranges.
What is the difference between Bigtable and BigQuery?
Bigtable is a database that serves requests efficiently through row keys and row-key ranges. BigQuery is an analytical warehouse for SQL queries over large datasets. A system can use Bigtable for application reads and BigQuery for historical analysis.
Is Google Bigtable strongly consistent?
Reads and writes that one cluster serves are strongly consistent. Replication across clusters is asynchronous, so requests that use different clusters generally observe eventual consistency. State the routing policy and failover behavior. Do not assign one consistency label to every deployment.