Serverless Event-Driven Architecture on GCP

Course Overview

In the modern cloud, applications don’t just “run”—they “react”. In this course, you will learn to build Asynchronous, Reactive Systems that scale automatically to zero when quiet and to billions when busy. You will master the triad of Cloud FunctionsArcane GlossaryA serverless execution environment for building and connecting cloud services, allowing you to create small, single-purpose functions., Pub/SubArcane GlossaryPublish/Subscribe. A messaging pattern where senders (publishers) don't send messages directly to receivers, but to 'topics' instead., and Cloud Storage triggers.

Learning Objectives

• Design decoupled architectures using Pub/Sub MessagingArcane GlossaryPublish/Subscribe. A messaging pattern where senders (publishers) don't send messages directly to receivers, but to 'topics' instead..
• Implement Serverless Handlers for HTTP and System Events.
• Master IdempotencyArcane GlossaryDefinition not found in Grimoire. and Dead Letter Queue (DLQ)Arcane GlossaryDefinition not found in Grimoire. patterns.
• Build real-time data pipelines that derive state from events.

Technical Deep Dive: The Pub/Sub Heartbeat

Google Cloud Pub/Sub is the distributed backbone of GCP.

• Topics: The “Radio Station” where messages are published.
• Subscriptions: The “Listeners” that receive messages.
• Pull vs Push: Push subscriptions (Standard for Functions) deliver messages via HTTP POST, ensuring real-time reaction without polling.

---

Walkthrough: The “Image Intel” Pipeline

Step 1: Initialize the Messaging Hub

Create a topic that will handle image processing notifications.

gcloud pubsub topics create image-master-topic
gcloud pubsub subscriptions create image-master-sub --topic image-master-topic

Step 2: The Event-Driven Logic

Create index.ts for a Cloud Function that reacts to image uploads.

import * as functions from '@google-cloud/functions-framework';
import { Storage } from '@google-cloud/storage';

const storage = new Storage();

functions.cloudEvent('processImage', async (cloudEvent: any) => {
  const file = cloudEvent.data;
  console.log(`📡 Event Detected: File ${file.name} uploaded to ${file.bucket}`);

  if (file.contentType?.startsWith('image/')) {
    // Simulate AI Image Intelligence
    console.log(`🧠 Processing image: ${file.name}...`);
    // Here you would call Google Vision AI or a custom ML model
  }
});

Step 3: Deployment with Trigger

Deploy the function and link it directly to a Cloud Storage bucket event.

gcloud functions deploy image-processor \
  --runtime nodejs22 \
  --trigger-resource my-arcane-bucket \
  --trigger-event google.storage.object.finalize \
  --entry-point processImage \
  --region us-central1

---

Advanced: Reliability Rituals

• The DLQ (Dead Letter Queue): If a function fails 5 times, move the message to a “Morgue” topic for human investigation.
• Idempotency: Use the eventId in the CloudEvent metadata to ensure your function doesn’t process the same image twice if there’s a retry.
• Tracing: Instrument your pipeline with Cloud Trace to visualize the latency between upload and finished processing.

Capstone Project: The Real-Time Insight Engine

Build a 3-stage event pipeline.

1. Stage 1: An HTTP Function receives raw JSON data and publishes it to a Pub/Sub topic.
2. Stage 2: A Background Function listens to the topic, validates the data, and stores it in Firestore.
3. Stage 3: A third Function reacts to the Firestore write to trigger a notification (e.g., Slack or Email).

---

The silence of the cloud is only the calm before the react. Build systems that never sleep.