Rename VLLM to VLLMOffline and create server-based VLLM/AsyncVLLM

[RobinPicard]

Addresses #1548

For the moment, this PR only creates the vllm server models and modifies the generator

[cpfiffer]

Do you have an example of use here?

[RobinPicard]

Do you have an example of use here?

I was using the script below to test it @cpfiffer

Script

import asyncio
from openai import AsyncOpenAI, OpenAI
from outlines.models import from_vllm_server
from outlines.generator import Generator

from pydantic import BaseModel


class Message(BaseModel):
    role: str
    content: str


def sync_f():
    client = OpenAI(
        base_url="http://0.0.0.0:8000/v1",
    )
    model = from_vllm_server(client)

    generator = Generator(model, output_type=Message)

    result = generator(
        "Hello, world!",
        model="facebook/opt-125m",
        extra_body={"chat_template": "{% for message in messages %}{% if message['role'] == 'user' %}{{ message['content'] }}{% endif %}{% endfor %}"},
        max_tokens=10,
    )
    print("--------------- sync -----------------")
    print(result)
    print("--------------- sync streaming -----------------")
    for result in generator.stream(
        "Hello, world!",
        model="facebook/opt-125m",
        extra_body={"chat_template": "{% for message in messages %}{% if message['role'] == 'user' %}{{ message['content'] }}{% endif %}{% endfor %}"},
        max_tokens=10,
    ):
        print(result)


def async_f():
    client = AsyncOpenAI(
        base_url="http://0.0.0.0:8000/v1",
    )
    model = from_vllm_server(client)

    generator = Generator(model, output_type=Message)

    async def generate():
        result = await generator(
            "Hello, world!",
            model="facebook/opt-125m",
            extra_body={"chat_template": "{% for message in messages %}{% if message['role'] == 'user' %}{{ message['content'] }}{% endif %}{% endfor %}"},
            max_tokens=10,
        )
        print(result)
    
    print("--------------- async -----------------")
    asyncio.run(generate())

    async def generate_stream():
        # Use async for directly on the generator.stream() method
        async for result in generator.stream(
            "Hello, world!",
            model="facebook/opt-125m",
            extra_body={"chat_template": "{% for message in messages %}{% if message['role'] == 'user' %}{{ message['content'] }}{% endif %}{% endfor %}"},
            max_tokens=5,
        ):
            await asyncio.sleep(0.1)
            print(result)

    print("--------------- async streaming -----------------")
    asyncio.run(generate_stream())


if __name__ == "__main__":
    sync_f()
    async_f()

[rlouf]

The integration of sync/async is very elegant. The only (minor) thing that bothers me is that, because we also support the offline inference we have to call the function from_vllm_server. I'm unsure how many people will be using the offline version, and I'm wondering if we should drop it if anything to not have to maintain something people are not going to be using.

[RobinPicard]

I agree that it's annoying to have to call it from_vllm_server. I can't tell you how popular vLLM offline is, but I think we also have the option of calling vLLM server just VLLM and rename the current VLLM into VLLMOffline. That way what is the most popular and what we want to put forward has the most straightforward name (I also find it a bit counter-intuitive to call "server" what's actually the client to a remote server)

[rlouf]

but I think we also have the option of calling vLLM server just VLLM and rename the current VLLM into VLLMOffline.

Let's do that.

[cpfiffer]

Unrelated to this PR, but how does LLM work here?

[rlouf]

The PR looks good, but we have a new unrelated error in the test workflow.