6. GCP Core Infrastructure: Storage

Introduction

Every application needs to store data, whether it's streamed media, user documents, or sensor readings. Google Cloud offers a robust suite of storage solutions to address the diverse needs of modern applications. In this article, we'll explore Google Cloud's five major storage products: Cloud Storage, Cloud SQL, Cloud Spanner, Firestore, and Cloud BigTable.

Storage classes

There are 3 primary storage classes in cloud storage

1. Standard Storage: Frequent access, highest performance. Ideal for web content, mobile apps, frequently used data.

2. Nearline Storage: Infrequent access (data you need maybe once a month). Lower cost than Standard, perfect for backups and less time-critical data.

3. Coldline Storage: Long-term archival (think once a year access). More restrictive retrieval than Nearline, but lowest cost.

Cloud Storage

Cloud Storage is Google's highly available, scalable, and fully managed object storage solution. It's designed for unlimited storage and frequent data access. Object storage excels at handling large, unstructured files where the entire file is typically retrieved at once rather than in small chunks. Typical use cases include serving website content, delivering videos, storing data for backup and disaster recovery, or large-scale archiving.

Object storage manages data as units called 'objects', each containing the data itself, relevant metadata (date created, author, etc.), and a globally unique identifier. Perfect for large unstructured datasets like videos, images, and audio files.

Cloud Storage organizes files into buckets, which require globally unique names. Strategically choose a bucket's geographic location to minimize latency - placing it closest to where your users or applications will access the data most frequently.

Versioning

Cloud Storage's default storage objects are immutable, so each modification creates a new version. With optional versioning enabled, Cloud Storage tracks changes throughout an object's history, providing a safeguard against accidental overwrites or deletions.

Lifecycle management

Because storing and retrieving large amounts of object dat can quickly become expensive, Cloud Storage also offers lifecycle management policies. These policies let you automate actions like:

• Deleting objects exceeding a certain age

• Transitioning less frequently used objects to lower-cost storage classes

• Moving objects to more restrictive (and low-cost) archiving after designated criteria are met.

Cloud SQL

Cloud SQL offers fully managed relational databases, including MySQL, PostgreSQL, and SQL server as a service. It is designed to hand off mundane, but necessary and often time-consuming tasks to Google (applying OS patches and updates, managing backups) so users can focus on building great applications instead. Cloud SQL doesn't require any software installation or maintenance and it can scale up to 128 processor cores, 864GB or RAM and 64TB of storage.

Cloud SQL supports managed backups so backed-up data is securely stored and accessible if a restore is required. The cost of an instance covers seven backups. Cloud SQL encrypts customer data when on Google's internal networks and when stored in database tables, temporary files, and backups. It also includes a network firewall, which controls network access to each database instance.

A benefit of cloud SQL instances is that they are accessible by other Google Cloud services and even external services.

Cloud Spanner

Cloud Spanner is a powerful, globally distributed relational database designed for applications demanding high transaction rates and ultra-consistent data across vast geographic distances. Think of it as a relational database (SQL, joins, etc.) that seamlessly scales to serve users spread worldwide, while guaranteeing your data is always up-to-date no matter where they are.

Example Use Cases: Global banking systems, multiplayer game backends, inventory systems spanning warehouses across continents.

Firestore

Firestore is a flexible NoSQL document database ideal for mobile, web, and server-side development. Unlike traditional databases that use tables and rows, Firestore stores data in 'documents' (containing key-value pairs), grouped into 'collections'. This structure excels when your application's data model frequently evolves.

A 'products' collection in Firestore could hold documents where each document represents a product with key-value pairs like:

• price: 9.99

• description: "A must-have gadget"

• inStock: true

Firestore queries are powerful for retrieving specific documents or sets fitting your criteria. Real-time synchronization keeps data up-to-date across connected devices, making it great for collaborative apps. And its built-in scaling means less time configuring infrastructure and more on building!

Firestore offers a generous free tier for getting started. To explore pricing details, consult Google's Firestore pricing page.

Cloud Bigtable

Cloud BigTable is Google's powerhouse NoSQL database built for truly massive datasets. Think petabytes of data (that's thousands of terabytes). It's the backbone of Google's own high-performance services like Search and Maps, making it a trusted choice when speed and handling sheer volume are critical.

Ideal Use Cases:

• Analyzing vast amounts of IoT sensor data in real-time for trends

• Tracking millions of financial transactions, requiring up-to-the-second accuracy

• Storing years of user behavior for complex website personalization

If you're working with smaller datasets or don't need lightning-fast response at such an enormous scale, other GCP storage options may be a better fit.

Comparing the storage options