Making Data Useful and Accessible

Study guide for Cloud Digital Leader, Unit 2 · Topic 3. This is the topic's lecture in reading form — every slide's teaching, figures and worked examples, in order — followed by the official Google Cloud pages its claims rest on.

What the exam guide asks. Discuss how smart analytics, business intelligence tools, and streaming analytics can add value in different business use cases.

Objectives, quoted from the exam guide:

Describe how Looker democratizes access to data by empowering individuals to self-serve business intelligence and create insights.
Discuss the value of analyzing and visualizing data from BigQuery in Looker to create real-time reports, dashboards, and integrating data into workflows.
Describe how streaming analytics in real time makes data more useful and generates business value.
Describe the main Google Cloud products that modernize data pipelines, including Pub/Sub and Dataflow.

Making data useful and accessible

Who can ask the data a question, and how soon

Who can ask — Looker lets people who do not write SQL self-serve answers. What they get from BigQuery — reports, dashboards, real-time insight, and data pushed into the tools they already use. How soon — streaming analytics, and the two products that carry the stream: Pub/Sub and Dataflow.

The last topic of Unit 2 is about the moment data turns into a decision, and the guide's summary names three ways it happens: smart analytics, business intelligence tools, and streaming analytics. The four objectives split cleanly into two questions. The first is who can ask the data a question. Business intelligence, or BI, used to mean waiting for an analyst to write a report; Looker's promise is that people who do not write structured query language, or SQL, can get their own answers, and the second objective asks what that looks like on top of BigQuery. The second question is how soon. Streaming analytics processes data as it arrives rather than in nightly batches, and the fourth objective names the two Google Cloud products that make a pipeline streaming: Pub/Sub, which receives and distributes the events, and Dataflow, which transforms them. Hold both questions — who, and how soon — and every objective in this topic is one of them.

Looker: business intelligence people can serve themselves

Analysts define the data once; everyone else asks it questions

Analysts describe the data once, in LookML, as a semantic model
Business users build queries in the Explore interface, without SQL
Looker's SQL generator turns each question into SQL for the database
One definition of each metric, reused by every report built on it

Worked example (synthetic). A retailer's regional managers each want margin by store. Instead of filing five report requests, they open the same Explore, pick store and margin, and get answers built on one agreed definition of margin.

The first objective asks how Looker democratizes access to data by letting individuals self-serve business intelligence, or BI. Start with what Looker is: Google calls it a product that helps you explore, share and visualize your company's data so that you can make better business decisions. The self-service comes from a division of labour, and it is worth knowing exactly where the line is. On one side, data analysts use LookML — short for Looker Modeling Language — to create and maintain data models that define the data structure and business rules for the data being analyzed. On the other, business users use the Looker query builder, or the Explore interface, to create queries based on that model. Between them sits the piece that makes it work: the Looker SQL generator translates LookML into structured query language, or SQL, which lets business users query without writing any LookML or SQL. The result is that a business user can build complex queries while focusing only on the content they need, not the complexities of SQL structure. There is a second benefit that is easy to miss. Because analysts write each SQL expression once, in one place, every report that uses a metric uses the same definition of it. Google puts it as two promises: Looker offers a unified surface to access all of an organization's data, and it lets you create consistent data models on top of all your data. Democratized access is not everyone writing their own numbers; it is everyone asking questions of the same numbers.

Where the SQL comes from

The business user never writes it; the model does

Loading Diagram...

Figure 1 — Mermaid diagram

Figure: A flow chart. A data analyst writes a LookML model once, holding structure and business rules. A business user picks fields in the Explore interface. Both feed the Looker SQL generator, which sends SQL to a database such as BigQuery; the formatted results return to the Explore interface.

Worked example (synthetic). An exam item says a sales director builds her own report without knowing SQL and asks what makes that possible. The graded answer is the analyst-built LookML model plus the SQL generator — not the director learning to query.

Drawn out, self-service business intelligence, or BI, is two inputs meeting in one place. On the left, a data analyst writes the LookML model once — the structure of the data and the business rules for it. Google describes LookML as the language used in Looker to create semantic data models. On the right, a business user picks fields in the Explore interface. Both arrive at the Looker SQL generator, which is the only thing in the picture that writes structured query language, or SQL. Google describes the round trip exactly: when a user creates a query, it is sent to the SQL generator; the SQL query is executed against the database, and then Looker returns the formatted results to the user in the Explore interface. Notice what the figure does not contain: an arrow from the business user to the database. That missing arrow is the whole objective. The user has direct access to answers without direct access to SQL, and the model in the middle is what keeps those answers consistent.

Looker on BigQuery: what the pairing adds

Reports, speed, real time, and data that goes where work happens

Capability	What Google says it gives you
Reports and dashboards	LookML tells Looker how to query, so everyone can create easy-to-read reports and dashboards
Speed	BI Engine is an in-memory service that accelerates queries, including those behind dashboards
Real time	BigQuery streaming supports continuous ingestion and analysis; Looker gives real-time insight from it
Alerts	A condition in the data, when met or exceeded, notifies chosen recipients
Workflows	Scheduled deliveries, and actions that send content to third-party services

Worked example (synthetic). A logistics firm streams delivery scans into BigQuery. Its operations dashboard in Looker shows late parcels as they happen, an alert tells the depot lead when a route crosses a threshold, and the daily summary lands in the team's chat channel.

The second objective asks for the value of analyzing and visualizing BigQuery data in Looker, and names three results: real-time reports, dashboards, and integrating data into workflows. Start with the pair. BigQuery is Google's fully managed data platform with business intelligence among its built-in features; Looker is an enterprise platform for business intelligence, data applications and embedded analytics. Google is careful to say you don't need Looker to use BigQuery — the pairing is a choice, and this table is the case for it. Reports and dashboards come from the model: LookML tells Looker how to query data, so everyone in the organization can create easy-to-read reports and dashboards. Speed comes from BigQuery BI Engine, a fast, in-memory analysis service that accelerates many structured query language, or SQL, queries, including queries used for BI dashboards. Real time comes from the data arriving continuously: BigQuery streaming supports continuous data ingestion and analysis, and Google's own reference pipeline ends with Looker providing real-time BI insights from the data stored in BigQuery. The last two rows are the workflow half of the objective. Alerts let you specify conditions in your data that, when met or exceeded, trigger a notification to specific recipients. And Looker can schedule immediate or recurring deliveries of dashboards, including to third-party services integrated with Looker, such as Slack.

From looking at data to acting on it

A dashboard someone must remember to open is only half the value

Figure. Three cards in a row: See, a fast dashboard on BigQuery data; Notice, an alert that tells people when a condition is met; Act, scheduled deliveries and actions that push content into other services.

Worked example (synthetic). Asked why a retailer's stock-out dashboard 'is not being used', the graded fix is an alert or a scheduled delivery to the store team — not a faster dashboard.

This figure is here because the phrase integrating data into workflows is the part of the objective candidates skip. A dashboard delivers value only when someone opens it. Looker closes that gap in two moves. The first is noticing: alerts are set on dashboard tiles, and Looker checks at a set frequency whether the condition has been met or exceeded and, if so, notifies the recipients. The second is acting where the work happens: beyond Looker's built-in destinations, you can use actions — also called integrations — to deliver content to third-party services that are integrated with Looker through an action hub server. So the progression runs see, notice, act. The same idea exists outside Looker, too: Connected Sheets brings the scale of BigQuery to the familiar Google Sheets interface — but that is BigQuery meeting a spreadsheet, not Looker, and the exam will expect you to tell them apart.

Streaming analytics: acting while the data is fresh

Continuous, not batched — because some data goes stale

Streaming analytics processes records continuously, not in batches
Batches bring long latency; time-sensitive data goes stale waiting
Real-time actions need continuous processing and analysis
Value: act on a click, a transaction or a reading as it happens

Worked example (synthetic). A bank scores card transactions overnight and learns of a fraud pattern the next morning. Streamed, the same analysis flags the pattern while the card is still being used.

The third objective asks how streaming analytics in real time makes data more useful and generates business value, and Google's definition already contains the argument. Streaming analytics is the processing and analyzing of data records continuously rather than in batches. It suits sources that send data in small pieces in a continuous flow as the data is generated — telemetry from connected devices, log files from web applications, ecommerce transactions, or information from social networks and geospatial services. The contrast is batch processing, which often processes large volumes of data at the same time, with long periods of latency. Google is fair to batch: it can be an efficient way to handle large volumes of data. But it does not work with time-sensitive data, because that data can be stale by the time it is processed. That is the business case in one word — stale. A price that should have changed an hour ago, or an alert about a transaction that has already cleared, has lost most of its value. Google states the requirement directly: generating real-time actions requires continuous stream processing and analysis. And it states the payoff: stream analytics makes data more organized, useful and accessible from the instant it is generated.

Batch against streaming

Choose by how fast the data loses its value

Question	Batch	Streaming
When is data processed?	Large volumes at the same time	Record by record, continuously
How long until a result?	Long periods of latency	As the data is generated
What does it suit?	Large volumes where a delay costs little	Small pieces in a continuous flow; time-sensitive data
What is it used for?	Efficient bulk processing	Real-time aggregation and correlation, filtering, or sampling

Worked example (synthetic). Monthly payroll is a batch problem: nothing is lost by waiting for the month to close. A delivery van's location is a streaming problem: an hour-old position is useless to the customer tracking it.

The exam rarely asks you to define streaming; it asks you to choose it, and this table is the choosing rule. Batch processes large volumes of data at the same time, with long periods of latency, which Google calls an efficient way to handle large volumes. Streaming processes records continuously, as the data is generated, and suits sources that send small pieces in a continuous flow. The deciding question is the third row: how fast does this data lose its value? If waiting costs little, batch is fine and often cheaper to reason about. If the data is time-sensitive, batch fails outright, because the data can be stale by the time it is processed. The last row lists what streaming is typically used for — real-time aggregation and correlation, filtering, or sampling. Neither is better in general. The graded answer is always the one that matches how perishable the data in the scenario is.

Three businesses, three kinds of value

Revenue, risk and relevance — each from acting sooner

Figure. Three cards: ecommerce, where streamed clickstreams drive real-time pricing, promotions and inventory; financial services, where streamed account activity reveals anomalous behavior and raises a security alert; news media, where streamed clicks are enriched to serve relevant articles.

Worked example (synthetic). An item describes a bank that wants to stop suspicious transfers before they settle and asks which capability delivers it. Streaming analytics with anomaly detection is the answer; a nightly report is the distractor.

Google gives three worked use cases for streaming analytics, and they are worth holding as three different kinds of business value, because exam scenarios are usually one of them in disguise. In ecommerce the value is revenue: analyze user clickstreams to optimize the shopping experience with real-time pricing, promotions, and inventory management. In financial services the value is risk: analyze account activity to detect anomalous behavior in the data stream and generate a security alert for abnormal behavior. In news media the value is relevance: stream user click records from various platforms and enrich the data with demographic information to better serve articles that are relevant to the targeted audience. In all three, the same event analyzed tomorrow would be worth much less — the shopper has left, the money has moved, the story is old. That is the sense in which streaming makes data more useful: not more data, but data used while it still matters. And Google adds one accessibility point that ties back to the first objective: its stream analytics provisioning reduces complexity and makes stream analytics accessible to both data analysts and data engineers.

Pub/Sub and Dataflow: modernizing the pipeline

One carries the events; the other transforms them

Pub/Sub: asynchronous messaging that decouples producers from processors
Pub/Sub delivers each event to every service that reacts to it
Dataflow: unified stream and batch processing — read, transform, write
Both managed: Dataflow provisions and removes its own worker VMs

Worked example (synthetic). A game studio's servers publish match events to Pub/Sub without knowing who uses them. A Dataflow pipeline turns them into per-minute leaderboards; a separate service reading the same events flags cheating. Neither slows the servers down.

The fourth objective names the two products that modernize a data pipeline, and the exam separates candidates who can say what each one does from those who treat them as a pair. Pub/Sub is the messaging layer. Google defines it as an asynchronous and scalable messaging service that decouples services producing messages from services processing those messages. Publishers send events to Pub/Sub without regard to how or when they are to be processed, and Pub/Sub then delivers the events to all the services that react to them. The decoupling is the modernization: in systems communicating through remote procedure calls, publishers must wait for subscribers to receive the data, while the asynchronous integration in Pub/Sub increases the flexibility and robustness of the whole system. Latencies are typically on the order of a hundred milliseconds, and Google names streaming analytics and data integration pipelines as what it is used for. Dataflow is the processing layer: a Google Cloud service that provides unified stream and batch data processing at scale, used to create pipelines that read from one or more sources, transform the data, and write it to a destination. It is fully managed — Google manages all of the resources needed to run it — and it can autoscale by provisioning extra worker virtual machines, or VMs, or shutting some down when fewer are needed. So: Pub/Sub moves events and does not transform them; Dataflow transforms data and is not a message bus.

Google's reference pipeline, end to end

Ingest, transform, store, present — one product per stage

Loading Diagram...

Figure 2 — Mermaid diagram

Figure: A left-to-right flow chart of Google's reference pipeline: an external system sends events to Pub/Sub, which ingests them; Dataflow reads from Pub/Sub and transforms or aggregates the data; Dataflow writes to BigQuery, the data warehouse; Looker provides real-time business intelligence insights from BigQuery.

Worked example (synthetic). An item lists four products and asks which one ingests a stream of device events. Pub/Sub is the graded answer; Dataflow is the tempting one, but it reads from Pub/Sub rather than receiving the events itself.

This is the whole topic on one line, and it is Google's own figure, described in the Dataflow overview stage by stage. Pub/Sub ingests data from an external system. Dataflow reads the data from Pub/Sub and writes it to BigQuery, and during this stage Dataflow might transform or aggregate the data. BigQuery acts as a data warehouse, allowing data analysts to run ad hoc queries on the data. And Looker provides real-time business intelligence, or BI, insights from the data stored in BigQuery. Read it as four jobs, one product each: ingest, transform, store, present. Exam items on this objective usually hand you a job and ask for the product, and the two errors to avoid are giving Pub/Sub's job to Dataflow, or BigQuery's job to Looker. It is also the picture that ties the unit together — the first two objectives of this topic live at the right-hand end, and the streaming objectives live at the left.

Pub/Sub and Dataflow, side by side

What makes each one a modernization

Aspect	Pub/Sub	Dataflow
Job	Messaging: receives and distributes events	Processing: reads, transforms, writes data
What it replaces	Publishers waiting on subscribers over direct calls	Separate code for batch and streaming — one model covers both
Operations	Scalable, asynchronous service	Fully managed; autoscales its worker VMs
Openness and reach	Distributes each event to every reacting service	Built on open source Apache Beam; templates need no Beam knowledge

Worked example (synthetic). A team keeps two codebases, one for nightly batch jobs and one for a live feed of the same data. The Dataflow row is the fix: the same programming model serves both.

Side by side, each product modernizes something different. Pub/Sub's job is messaging, and what it replaces is waiting: publishers that had to wait for subscribers to receive the data now hand events to a service that delivers them asynchronously to every service that reacts to them. Dataflow's job is processing, and what it replaces is duplication: Dataflow uses the same programming model for both batch and stream analytics, and a solution built on it can grow with your needs as you move from batch to streaming. Operationally, Dataflow is fully managed and autoscales its worker virtual machines, or VMs. Two further facts are exam favourites. Dataflow is built on the open source Apache Beam project, which matters for portability. And Google provides templates for common scenarios that you can deploy without knowing any Apache Beam programming concepts — which is the self-service argument of objective one, arriving at the engineering end of the pipeline. One more detail is worth knowing because distractors misuse it: by default, Dataflow provides exactly-once processing of every record.

What this topic actually tests

Who can ask, how soon, and which product does which job

Who writes the SQL? the LookML model and the SQL generator, not the business user. Is a dashboard enough? alerts and deliveries put data into workflows. Batch or stream? ask how fast the data goes stale. Which product? Pub/Sub ingests, Dataflow transforms, BigQuery stores, Looker presents.

Close the topic on four discriminations. First, self-service: when a business user gets their own answers in Looker, the structured query language, or SQL, is written by the Looker SQL generator from a model analysts defined — that is what democratized access means. Second, value beyond the dashboard: Looker on BigQuery gives real-time reports and dashboards, but the objective also says integrating data into workflows, which is alerts, scheduled deliveries and actions. Third, batch against streaming: choose by how quickly the data goes stale, because streaming analytics processes records continuously and batch brings long periods of latency. Fourth, the pipeline: Pub/Sub ingests the events, Dataflow transforms them, BigQuery stores them for analysis, and Looker provides the insight. Unit 3 picks up where this line ends — what artificial intelligence and machine learning can do with data that is already clean, current and accessible.

Unit 2.3 study guide — Making Data Useful and Accessible