How Generative AI Generates Text — and How to Work with It

Text-based generative AI systems, often called large language models (LLMs),  can feel almost magical. You type a sentence, and within seconds, the LLM produces a polished paragraph. It explains complex topics. It drafts emails. It writes code.

But what is it actually doing?

And more importantly — how should we be using it? And when should we be cautious about relying on it?

Recently, I watched two Ed Tech Tea sessions that explored both sides of this question. The first session focuses on how generative AI works behind the scenes, and the second one is about how we can prompt it more effectively. Together, they offered a simple but important message: AI isn’t intelligent in the way we are — it’s predictive. And the better we understand that, the better we can use it.

How AI Really Generates Text

At its core, generative AI is a prediction machine.

It is trained on massive amounts of text collected from the internet. Before training, that text is cleaned and converted into numbers (because computers don’t “read” words, they process numerical representations of them). 

The model learns patterns in those numbers, patterns that correspond to how words tend to follow one another. During training, it repeatedly tries to predict the next word in a sentence, compares its guess to the actual word, and adjusts its internal parameters to improve.

When generating text, the model calculates which word has the highest probability of coming next based on those learned statistical patterns.

If you type:

“The Eiffel Tower is in…”

The system doesn’t “know geography.” It calculates which word is most statistically likely to come next based on patterns it has “seen” before. Usually, the answer is “Paris, France.”

Below is a picture illustrating how the LLM works for the above situation.

Every sentence it generates works the same way, predicting one word at a time.

This explains something important: AI does not understand meaning. There is no reason. It does not have beliefs. It produces output based on probability.

That’s why it can sound confident even when it’s incorrect. It generates language in the style of the text it was trained on, and much of human writing expresses information assertively, even when uncertainty exists.

The first session also reminded us that AI systems depend on enormous infrastructure: electricity, computing power, water, and often invisible human labor. People label data, filter harmful content, and train systems — sometimes under difficult conditions. AI may feel automated, but it is deeply connected to human systems.

Why Prompting Matters

If an LLM is just predicting words, then prompting becomes crucial.

The system only works with what you give it. If your prompt is vague, the output will likely be vague. If your instructions are structured and clear, the output improves.

One of the biggest takeaways from the prompting session was that improving prompts isn’t about secret tricks — it’s about clarity and structure.

For example, instead of writing:

“Help me with my essay.”

You might write:

Task:

• I’m writing a 5-page essay about the impact of social media on teenagers’ mental health.
  
• My professor said my first draft was too descriptive and didn’t have a strong argument.
  

Context:

• The assignment asks us to analyze social media’s impact, not just describe it.
  
• We need to use at least three academic sources.
  

What I need:

1. Help me develop a clearer thesis statement.
   
2. Suggest ways to structure my argument so it’s more analytical.
   
3. Point out where I might need stronger evidence or counterarguments.
   

Using simple formatting, like headings, bullet points, or numbered lists, can help clarify your thinking by forcing you to define what you actually want. While this doesn’t change the AI itself, it often improves the quality of your output by making your ideas more organized and intentional.

Working with AI, Not Expecting Perfection

Another helpful strategy discussed was using AI in iterative workspaces (like Canvas in Gemini, Canvas in ChatGPT, or Artifacts in Claude). Instead of expecting a perfect answer in one try, you can:

• Generate a draft
  
• Highlight a section
  
• Ask for revisions
  
• Refine tone
  
• Adjust structure
  

This shifts AI from being a “one-click answer machine” to a collaborative drafting tool.

The key is remaining an active participant in the process — reviewing, revising, and shaping the output rather than accepting it passively.  AI should not replace thinking, it should support it.

Using AI as a Tool, Not an Authority

Understanding how generative AI works changes how we relate to it.

Because it predicts rather than understands, here is an example.

The diagram of the “LLM Training: Essential Component” was generated from a simple prompt asking for the main components involved in training a large language model. At first glance, most of the elements make sense.

But one label stands out: “War (Operations).” That is something we did not explicitly ask for. Why might the model include it?

One possible explanation is that the model associates large-scale technology systems with government or military infrastructure because those ideas frequently appear together in the data it was trained on. The model is not reasoning about whether “war” logically belongs in this diagram. Instead, it is predicting words and concepts that often appear near discussions of large technological systems.

This example highlights an important point: generative AI does not truly understand the topics it writes about. It generates outputs by predicting likely patterns in language. Most of the time this works surprisingly well, but sometimes the model adds ideas that were never part of the original request.

Prompt to Gemini: I need a diagram that captures the components needed to train an LLM: water, electricity, computing power, amount of information, and algorithms

we should:

• Double-check factual claims
  
• Avoid treating the LLM as an expert
  
• Be cautious with complex or sensitive topics
  
• Recognize the LLM’s limitations
  

When we are experts in a specific subject area, an LLM can be incredibly useful for brainstorming, outlining, summarizing, and drafting, especially when we provide thoughtful prompts in that particular field. However, when we are still learning about a subject area, using generative AI for brainstorming or outlining will take away our original thinking and creativity, and will diminish our learning process.

AI literacy, then, isn’t just about knowing how to use the tool. It’s about understanding what it is, and what it isn’t.

Final Thought

Generative AI feels intelligent because it produces fluent, confident language. But fluency is not the same as understanding.

As we saw earlier, these systems generate text by predicting patterns in enormous amounts of data. That ability makes them powerful tools for drafting, brainstorming, and organizing ideas. At the same time, it also explains why they can produce errors, include unexpected information, or sound certain even when the content is wrong.

Recognizing both sides of this technology is essential. When we understand that AI is a prediction system rather than a thinking mind, we approach it differently. We write clearer prompts, question its outputs, and verify important information instead of accepting it automatically.

In the end, generative AI works best not as an authority, but as a tool that supports human thinking. The responsibility for reasoning, judgment, and learning still belongs to us.