Building an autonomous Quantitative Trading Agent for the Nifty 500 by self-hosting a 70B Teacher Model on AMD Instinct GPUs

Community Corner Showcase
, ,
1

Hello everyone! I’m a student developer currently building an open-source quantitative trading agent for the Indian Nifty 500 index. I wanted to share my project architecture with the community and see if anyone has insights on optimizing inference for strict financial data schemas.

The Problem: Most autonomous trading agents rely on rigid, math-only technical indicators or incredibly expensive, tightly rate-limited closed APIs (like OpenAI’s) to provide complex reasoning for market entries, stop-losses, and targets.

The Solution & AMD Integration: To bypass API limitations entirely, I am building a “Teacher-Student” distillation pipeline. I am leveraging the AMD Developer Cloud (Instinct GPUs) and the ROCm stack to completely self-host a massive open-source reasoning model (like Llama-3.3-70B-Instruct). This 70B AMD-powered “Teacher” model autonomously ingests historical OHLCV candlestick data and outputs perfect, hindsight financial rationales strictly formatted in JSON.

By using my AMD instance, I am generating a high-quality Supervised Fine-Tuning (SFT) dataset containing thousands of stock samples with zero rate-limits. Finally, I use that AMD-generated dataset to fine-tune a much smaller, hyper-fast local model (google/gemma-4-4b) using Unsloth.

Open Questions for the Community:

  1. For those using Ollama or vLLM on ROCm, do you have any tips for maximizing tokens/sec when generating strictly formatted JSON structures?

  2. Has anyone tried running continuous, multi-hour “batch” inference loops entirely on MI210/MI250 instances?

Looking forward to hearing from you all and sharing my final Github repo when the hackathon resolves!

2

Hello,
Using “Teacher-Student” method to skip expensive APIs is genius, as it makes your trading agent much more affordable and private. AMD MI250 power lets you generate huge amounts of data, but a potential downside is that forcing strict JSON can sometimes slow down your inference speed or lead to formatting errors. To solve this, I suggest using guided decoding libraries like Outlines or XGrammar with vLLM, which will force the model to follow your JSON schema perfectly without wasting extra tokens or compute time. Thanks for sharing, if you will need anything please do not hesitate.

3

thank you currenlty buidling a Distillation pipline and preparing a dataset.