{"id":30,"date":"2026-04-08T14:57:02","date_gmt":"2026-04-08T13:57:02","guid":{"rendered":"https:\/\/shopping-news.site\/?p=30"},"modified":"2026-04-08T15:16:11","modified_gmt":"2026-04-08T14:16:11","slug":"chat-templates-can-improve-lm-inferencing","status":"publish","type":"post","link":"https:\/\/shopping-news.site\/?p=30","title":{"rendered":"Chat Templates can improve LM inferencing."},"content":{"rendered":"
\n

The topic Chat Templates can improve LM inferencing.<\/strong> is currently the subject of lively discussion \u2014 readers and analysts are keeping a close eye on developments.<\/p>\n

This is taking place in a dynamic environment: companies\u2019 decisions and competitors\u2019 reactions can quickly change the picture.<\/p>\n

We load the SmolLM3 tokenizer and see its tokenizer.chat_template. When the user asks \u201cHow far is the sun?\u201d, the template produces a single prompt like:<\/p>\n

The model then fills in the assistant\u2019s reply. for example, the output might be: \u201cThe sun is about 93 million miles away.\u201d<\/p>\n

Imagine you have a talking robot and you want it to know who\u2019s talking and what to say. A chat template is like a script or recipe that tells the robot how to set up each conversation.<\/p>\n

Every time someone asks something, the computer fills in the blanks, and the puppet (the AI) knows exactly when to speak.<\/p>\n

This way, the conversation always follows the same \u201crecipe\u201d or \u201cscript,\u201d so the AI doesn\u2019t get confused about what to do. This helps even small models (like SmolLM3) understand the pattern of a conversation.<\/p>\n

In other words, a chat template defines the format of the prompt. for example, a template might say:<\/p>\n

\"\"<\/figure>\n

Here, {{system_message}} is a placeholder for any initial instruction, {{user_message}} is where the user\u2019s input goes, and {{assistant_message}} is where the model should reply. The tokenizer takes this template and the conversation data, fills in the placeholders, and then tokenizes the resulting string. Hugging Face\u2019s Transformers library handles this automatically, so developers don\u2019t have to manually concatenate strings or add special tokens.<\/p>\n

Placeholders: These are like blanks in the script. Common placeholders are {{user}}, {{assistant}}, and sometimes {{system}} or others. When you run a conversation, the actual text from the user or previous assistant message is plugged into these placeholders. for example, if {{user}} was \u201cWhat is 2+2?\u201d, the template might produce something like \u201cUser: What is 2+2?\u201d in the final prompt.<\/p>\n

Roles: Templates usually distinguish who is speaking. The most common roles are:<\/p>\n

Some templates might also include Function\/Tool messages (for advanced uses), but basic chat templates focus on system\/user\/assistant.<\/p>\n

`, if needed) and converts it to IDs. for example, if the system message is \u201cSolve math problems,\u201d and the user asks \u201c2+2?\u201d, the filled prompt might be:<\/p>\n

These features can improve accuracy and safety because the model sees prompts exactly as the trainers intended. As Hugging Face puts it, chat templates are the foundation of instruction tuning, giving a stable format.<\/p>\n

\"\"<\/figure>\n

In summary, chat templates greatly simplify formatting, but you must still handle safety (e.g. sanitize inputs), manage long conversations, and ensure the template itself is correct for your use case.<\/p>\n

Templates let you quickly answer FAQs or store snippets for re-use.<\/p>\n

Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment’s permalink.<\/p>\n

For further actions, you may consider blocking this person and\/or reporting abuse<\/p>\n

Thank you to our Diamond Sponsors for supporting the DEV Community<\/p>\n

Google AI is the official AI Model and Platform Partner of DEV<\/p>\n

DEV Community \u2014 A space to discuss and keep up software development and manage your software career<\/p>\n

Built on Forem \u2014 the open source software that powers DEV and other inclusive communities.<\/p>\n

We’re a place where coders share, stay up-to-date and grow their careers.<\/p>\n