ChatWriter

This notebook provides a quick overview for getting started with Writer chat models.

Writer has several chat models. You can find information about their latest models and their costs, context windows, and supported input types in the Writer docs.

:::

Overview

Integration details

Class	Package	Local	Serializable	JS support	Package downloads	Package latest
ChatWriter	langchain-community	❌	❌	❌	❌	❌

Model features

Tool calling	Structured output	JSON mode	Image input	Audio input	Video input	Token-level streaming	Native async	Token usage	Logprobs
✅	❌	❌	❌	❌	❌	✅	✅	✅	❌

Setup

To access Writer models you'll need to create a Writer account, get an API key, and install the writer-sdk and langchain-community packages.

Credentials

Head to Writer AI Studio to sign up to OpenAI and generate an API key. Once you've done this set the WRITER_API_KEY environment variable:

import getpass
import os

if not os.environ.get("WRITER_API_KEY"):
    os.environ["WRITER_API_KEY"] = getpass.getpass("Enter your Writer API key:")

Installation

The LangChain Writer integration lives in the langchain-community package:

%pip install -qU langchain-community writer-sdk

[notice] A new release of pip is available: 24.2 -> 24.3.1
[notice] To update, run: pip install --upgrade pip
Note: you may need to restart the kernel to use updated packages.

Instantiation

Now we can instantiate our model object and generate chat completions:

from langchain_community.chat_models.writer import ChatWriter

llm = ChatWriter(
    model="palmyra-x-004",
    temperature=0.7,
    max_tokens=1000,
    # other params...
)

API Reference:ChatWriter

Invocation

messages = [
    (
        "system",
        "You are a helpful assistant that writes poems about the Python programming language.",
    ),
    ("human", "Write a poem about Python."),
]
ai_msg = llm.invoke(messages)

print(ai_msg.content)

In realms of code, where logic weaves and flows,
A language rises, Python by its name,
With syntax clear, where elegance it shows,
A serpent, wise, that time and space can tame.

Born from the mind of Guido, pure and bright,
Its beauty lies in simplicity and grace,
A tool of power, yet gentle in its might,
In every programmer's heart, a cherished place.

It dances through the data, vast and deep,
With libraries that span the digital realm,
From machine learning's secrets to keep,
To web development, it wields the helm.

In the hands of the novice and the sage,
Python spins the threads of digital dreams,
A language that can turn the age,
With a gentle learning curve, its appeal gleams.

It's more than code, a community it builds,
Where knowledge freely flows, and all are heard,
In Python's world, the future unfolds,
A language of the people, for the world.

So here's to Python, in its gentle might,
A master of the modern coding art,
May it continue to light our path each night,
In the vast, evolving world of code, its heart.

Streaming

ai_stream = llm.stream(messages)

for chunk in ai_stream:
    print(chunk.content, end="")

In realms of code where logic weaves,
A language rises, Python, it breezes,
With syntax clear and simple to read,
Through its elegance, our spirits are fed.

Like rivers flowing, smooth and serene,
Its structure harmonious, a coder's dream,
Indentations guide the flow of control,
In Python's world, confusion takes no toll.

A vast library, a treasure trove so bright,
For web and data, it offers its might,
With modules and packages, a rich array,
Python empowers us to code in play.

From AI to scripts, in flexibility it thrives,
A language of the future, as many now derive,
Its community, a beacon of support and cheer,
With Python, the possibilities are vast, far and near.

So here's to Python, in its gentle grace,
A tool that enhances, a language that embraces,
The art of coding, with a fluent, flowing pen,
In the Python world, we code, and we begin.

Chaining

We can chain our model with a prompt template like so:

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are a helpful assistant that writes poems about the {input_language} programming language.",
        ),
        ("human", "{input}"),
    ]
)

chain = prompt | llm
chain.invoke(
    {
        "input_language": "Java",
        "input": "Write a poem about Java.",
    }
)

API Reference:ChatPromptTemplate

AIMessageChunk(content='In the realm of code, where logic weaves and flows,  \nA language rises, like a phoenix from the code\'s throes.  \nJava, the name, a cup of coffee\'s steam,  \nBrewed in the minds, where digital dreams gleam.\n\nWith syntax clear, like morning\'s misty hue,  \nIn classes and objects, it spins a tale so true.  \nA platform agnostic, with a byte to spare,  \nAcross the devices, it journeys everywhere.\n\nInheritance and polymorphism, its power\'s core,  \nLike ancient runes, in every line they bore.  \nEncapsulation, a shield, with data it does hide,  \nIn the vast jungle of code, it stands as a guide.\n\nFrom applets small, to vast, server-side apps,  \nIts threads run swift, through the computing traps.  \nA language of the people, by the people, for the people’s use,  \nBuilt on the principle, "write once, run anywhere, with no excuse."\n\nIn the heart of Android, it beats, a steady drum,  \nCrafting experiences, in every smartphone\'s hum.  \nIn the cloud, in the enterprise, its presence is vast,  \nA cornerstone of computing, built to last.\n\nOh Java, thy elegance, thy robust design,  \nA language that stands, in any computing line.  \nWith every update, with every new release,  \nThy community grows, with a vibrant, diverse peace.\n\nSo here\'s to Java, the versatile, the grand,  \nA language that shapes the digital land.  \nMay it continue to evolve, to grow, to inspire,  \nIn the endless quest of turning thoughts into digital fire.', additional_kwargs={}, response_metadata={'token_usage': {'completion_tokens': 345, 'prompt_tokens': 33, 'total_tokens': 378, 'completion_tokens_details': None, 'prompt_token_details': None}, 'model_name': 'palmyra-x-004', 'system_fingerprint': 'v1', 'finish_reason': 'stop'}, id='run-a5b4be59-0eb0-41bd-80f7-72477861b0bd-0')

Tool calling

Writer supports tool calling, which lets you describe tools and their arguments, and have the model return a JSON object with a tool to invoke and the inputs to that tool.

ChatWriter.bind_tools()

With ChatWriter.bind_tools, we can easily pass in Pydantic classes, dict schemas, LangChain tools, or even functions as tools to the model. Under the hood these are converted to tool schemas, which looks like:

{
    "name": "...",
    "description": "...",
    "parameters": {...}  # JSONSchema
}

and passed in every model invocation.

from pydantic import BaseModel, Field


class GetWeather(BaseModel):
    """Get the current weather in a given location"""

    location: str = Field(..., description="The city and state, e.g. San Francisco, CA")


llm_with_tools = llm.bind_tools([GetWeather])

ai_msg = llm_with_tools.invoke(
    "what is the weather like in New York City",
)

AIMessage.tool_calls

Notice that the AIMessage has a tool_calls attribute. This contains in a standardized ToolCall format that is model-provider agnostic.

print(ai_msg.tool_calls)

[{'name': 'GetWeather',
  'args': {'location': 'New York City, NY'},
  'id': 'chatcmpl-tool-fe70912c800d40fc8700d604d4823001',
  'type': 'tool_call'}]

For more on binding tools and tool call outputs, head to the tool calling docs.

API reference

For detailed documentation of all Writer features, head to our API reference.

Related

• Chat model conceptual guide
• Chat model how-to guides