Getting started with vLLM for a small tool-using AI agent

I’m a computer science student at Afeka working on AI and agent projects, and I’m exploring AMD Developer Cloud for a first GPU experiment.

I’d like to serve an open model with vLLM and connect it to a simple tool-using agent. For a beginner, what’s the smoothest path from a preconfigured notebook to an OpenAI-compatible endpoint? Which model and instance would you suggest for a small test, and how can I monitor usage and shut the GPU down so credits don’t keep running?

I’ve found the getting-started resources; I’m especially interested in the agent integration and cost-control steps. Thanks!

Hey! AMD’s guide is close to exactly what you’re describing: Deploying Hermes Agent for Free on AMD Developer Cloud. It uses a single MI300X droplet with the ROCm Software image, runs the vllm/vllm-openai-rocm Docker image, and serves Qwen3.5-35B-A3B with --enable-auto-tool-choice --tool-call-parser qwen3_coder, so any OpenAI-compatible agent can use http://:8001/v1.

One change I’d make: that guide opens the port with no key, so add --api-key to vllm serve, or reach the port over an SSH tunnel instead of opening it.

For cost, powering off isn’t enough. AMD’s Developer Cloud page says a powered-off GPU instance still bills, so destroy the droplet when you’re done. Your credit balance shows on the My AMD Home page in the console, and amd-smi monitor on the droplet shows live GPU use.