Introduction to Generative AI is Google's short introductory video and course on what generative AI is, how it is used, and how it differs from traditional machine learning. Watch the video below, then use this companion guide to go deeper on the core ideas such an introduction covers: the AI and machine learning hierarchy, generative versus discriminative models, large language models, prompts, hallucinations and responsible use. The last sections add something most summaries skip: what generative AI demands from hardware, with numbers you can check yourself.
Video: "Introduction to Generative AI", published on YouTube by the Google Cloud Tech channel.
Key takeaways
- Generative AI creates new content (text, images, audio, code, video) by learning the patterns and structure of its training data.
- It is a subset of deep learning, which is a subset of machine learning, which is a subset of artificial intelligence.
- Discriminative models learn to separate or label existing data; generative models learn what the data looks like well enough to produce new examples.
- Large language models generate text one token at a time by predicting the most likely next token, which is why they can sound confident and still be wrong.
- Running these models is limited as much by memory size and memory bandwidth as by raw compute, which is why generative AI is reshaping chip design.
About Google's Introduction to Generative AI course
Google offers Introduction to Generative AI on Google Skills, listed as an introductory microlearning course of about 45 minutes. The course page says it explains what generative AI is, how it is used and how it differs from traditional machine learning methods, and that it also covers Google tools for building your own generative AI applications. It is part of Google's Beginner: Introduction to Generative AI learning path, which the path page describes as running from the fundamentals of large language models to responsible AI principles. The video embedded above is Google's introductory video on the same topic.
The sections below are an independent companion written for this blog. They explain the concepts any good introduction to generative AI covers; they are not a transcript or summary of Google's video.
AI, machine learning, deep learning and generative AI
These terms are nested, and mixing them up is the most common beginner mistake.
| Term | What it means | Example |
|---|---|---|
| Artificial intelligence (AI) | The broad field of building systems that perform tasks we associate with human intelligence | A chess program, a route planner |
| Machine learning (ML) | A branch of AI where systems learn patterns from data instead of following hand-written rules | A spam filter trained on labelled emails |
| Deep learning | Machine learning with many-layered neural networks, able to learn from raw data such as pixels or text | Image recognition, speech recognition |
| Generative AI | Deep learning models that produce new content similar to their training data | A chatbot writing an email, a model generating an image from a description |
Supervised, unsupervised and self-supervised learning
- Supervised learning uses labelled examples (input and correct answer) and learns to predict the label for new inputs.
- Unsupervised learning uses unlabelled data and finds structure in it, such as clusters of similar customers.
- Self-supervised learning creates its own labels from the data. Language models are pre-trained this way: the "label" for each position in a sentence is simply the next word. This is what lets them learn from very large amounts of text without manual labelling.
Generative versus discriminative models
A discriminative model learns the boundary between classes: given an input, what is the most likely label? A generative model learns the distribution of the data itself, so it can produce new samples that look like the training data.
| Discriminative model | Generative model | |
|---|---|---|
| Question it answers | "Is this a cat or a dog?" | "What does a picture of a cat look like?" |
| Output | A label, number or probability | New content: text, image, audio, code |
| Examples | Classifiers, regression models, defect detectors | Large language models, diffusion image models |
| Typical use in engineering | Classifying wafer map patterns, predicting yield | Drafting scripts or documentation, generating test stimulus ideas |
A simple test: if the output is a number, a class or a probability, it is not generative AI. If the output is natural language, an image, audio or code, it usually is.
How large language models work
- Tokens. Text is split into tokens, which are words or pieces of words. Each token is mapped to a vector of numbers.
- The transformer. Most modern language models use the transformer architecture, introduced by researchers at Google in the 2017 paper "Attention Is All You Need". Its attention mechanism lets every token weigh the relevance of every other token in the context.
- Pre-training. The model reads a very large text corpus and learns to predict the next token. In doing so it absorbs grammar, facts and patterns of reasoning, along with errors and biases in the data.
- Tuning. The pre-trained model is then adapted with smaller, curated data sets, for example instruction tuning so that it follows requests, and feedback from human reviewers so that its answers are more helpful and safer.
- Generation. At run time the model predicts a probability for every possible next token, picks one (with some controlled randomness, often set by a "temperature" parameter), appends it, and repeats.
A foundation model is a large model pre-trained on broad data that can be adapted to many tasks. Large language models are the best-known kind; there are also foundation models for images, audio, code and combinations of these (multimodal models).
Types of generative model by input and output
| Type | Input | Output | Typical approach |
|---|---|---|---|
| Text to text | A prompt | An answer, summary, translation or code | Transformer language model |
| Text to image | A description | An image | Diffusion model guided by a text encoder |
| Text to audio or speech | Text | Speech or music | Transformer or diffusion based audio models |
| Text to video | A description | A short video clip | Diffusion or transformer models over frames |
| Text to action | A goal | A sequence of tool calls or steps | Language model connected to tools (often called an agent) |
Prompts, hallucinations and grounding
A prompt is the input you give a generative model. Clear prompts state the task, the context, the format you want and any constraints. For example, "Write a Verilog testbench for a 4-bit up counter with synchronous reset; use a 10 ns clock and print a FAIL message if the count is wrong after ten cycles" will produce far more useful output than "write a testbench".
A hallucination is fluent output that is false or unsupported, such as an invented citation or a wrong register name. It happens because the model generates plausible tokens; it has no built-in step that checks facts. Missing or outdated training data and vague prompts make it more likely.
Common ways to reduce hallucinations:
- Grounding: give the model the source material (a specification, a datasheet) and ask it to answer only from that material. Systems that fetch relevant documents automatically before generating are called retrieval-augmented generation (RAG).
- Ask for sources and check them.
- Constrain the output (a fixed format, a short answer, "say you do not know if the document does not say").
- Review by a person for anything that will be published, shipped or signed off.
Responsible use of generative AI
- Accuracy: treat output as a draft, never as a verified fact.
- Confidentiality: do not paste proprietary RTL, netlists, customer data or unreleased specifications into public tools unless your company has approved that tool for such data.
- Bias: models reflect their training data, including its gaps and biases.
- Intellectual property: check licence terms for generated code and content before using it in a product.
- Transparency: say when content was generated or assisted by AI where your organisation or audience expects it.
What generative AI asks of hardware: worked numbers
For a semiconductor audience, the most useful thing to understand is why generative AI is so hungry for memory. Two back-of-the-envelope rules explain a great deal.
Rule 1: memory for the weights = parameters × bytes per parameter
| Model size | 16-bit (2 bytes) | 8-bit (1 byte) | 4-bit (0.5 byte) |
|---|---|---|---|
| 7 billion parameters | 14 GB | 7 GB | 3.5 GB |
| 13 billion parameters | 26 GB | 13 GB | 6.5 GB |
| 70 billion parameters | 140 GB | 70 GB | 35 GB |
This counts the weights only (1 GB = 109 bytes). Real deployments also need memory for the key-value cache, which grows with context length and the number of simultaneous users, plus activations and runtime overhead. Training needs several times more again for gradients and optimiser state. This is why reducing precision (quantisation) matters so much for running models on laptops and phones.
Rule 2: generation speed is often limited by memory bandwidth
When a model generates text for a single user, each new token requires reading essentially all the weights from memory once. So a rough upper bound is:
tokens per second ≈ memory bandwidth ÷ size of the weights
- A 7B model at 4-bit (3.5 GB) on a device with 100 GB/s of memory bandwidth: about 100 ÷ 3.5 ≈ 29 tokens per second at best.
- The same model at 16-bit (14 GB) on an accelerator with 1,000 GB/s: about 71 tokens per second at best.
- A 70B model at 16-bit (140 GB) spread across accelerators with a combined 3,000 GB/s: about 21 tokens per second at best.
The bandwidth figures are round illustrative numbers, not specific products, and the bound ignores the key-value cache and compute limits. Batching many users together raises total throughput because the weights read from memory are reused across requests. These two rules explain why high-bandwidth memory, advanced packaging and on-device neural processing units are central to AI hardware. The AI model memory and bandwidth sizer on The Semiconductor Hub does the full calculation, including the cache, precision and parallelism.
Where to go next
- For a broader view of the technology and its uses, read what generative AI is and its potential.
- To see how this reaches personal computers, read what an AI PC is and how it differs from a conventional PC.
- For the chips and software behind on-device AI, read the analysis of the edge AI semiconductor and software ecosystem.
- For short definitions of AI terms, see the AI entry in The Semiconductor Hub glossary.
Test yourself
- A model reads an image of a wafer map and outputs "edge ring" or "centre cluster". Is it generative or discriminative?
- How much memory do the weights of a 13 billion parameter model need at 8-bit and at 16-bit precision?
- Estimate the best-case generation speed of that 13B model at 8-bit on a device with 200 GB/s of memory bandwidth, for a single user.
- Why does giving a language model the relevant datasheet section reduce hallucinations?
Answers: (1) Discriminative: it outputs a class label. (2) 13 GB at 8-bit, 26 GB at 16-bit. (3) About 200 ÷ 13 ≈ 15 tokens per second at best. (4) The model can copy facts from text in its context instead of relying on patterns memorised during training, and you can check the answer against the source.
Frequently asked questions
Is Google's Introduction to Generative AI course free?
The course is listed on Google Skills as a 45-minute introductory course, and the video is freely available on YouTube. Check the course page for current access terms, badges and any sign-in requirements.
What is the difference between generative AI and traditional machine learning?
Traditional machine learning models usually predict a label or a number from an input, such as classifying an email as spam. Generative AI models learn the structure of their training data well enough to create new content, such as writing the email.
What is a large language model?
A large language model is a neural network, usually a transformer, trained on very large amounts of text to predict the next token. After tuning, it can answer questions, summarise, translate and write code.
Why do generative AI models hallucinate?
They generate the most plausible continuation of the text, not a verified fact. Gaps in training data, outdated information and vague prompts make false but fluent answers more likely. Grounding the model in source documents and reviewing output reduce the risk.
Do I need to know programming to start learning generative AI?
No. Introductory courses such as Google's focus on concepts and use. Programming becomes useful when you want to build applications on top of models, connect them to your own data or evaluate them systematically.


Your comments will be moderated before it appears here.