NVIDIA 发布 Molt 强化学习框架，简化训练流程

NVIDIA just released a new RL framework.

Your entire setup is one Python file:

• write a class with a step() function - return a reward from it - the framework handles the rest, up to 1T-class MoE training

In a typical RL framework, the environment and reward logic live inside the trainer's codebase.

Adding a new task requires writing framework code, registering it, and wiring up config for it.

The reward usually comes from a separately trained reward model, which needs its own GPUs and placement next to the actor and critic.

NVIDIA's Molt framework drops both requirements.

Every run takes an agent_path flag pointing at one Python module, and the trainer code never changes.

The module gives you two ways to plug in an agent.

> The first is an Env class where you write a step() function that receives the model's output and returns a reward, and the framework handles everything else, i.e., calling the model, tokenizing, and looping turn after turn until the episode ends.

> The second is a ChatAgent class, for agents that already exist as their own loop. Here, you write the loop yourself, calling the model through a standard OpenAI or Anthropic client, and the framework spins up a local server so those calls hit its vLLM engines instead of an external API.

Either way, the trainer sees the same thing, i.e., a trajectory of tokens and a reward at the end.

That reward can come from a string-match grader, code executed in a sandboxed subprocess, or an LLM-as-judge call.

So there is no reward model to train or place, and no environment code living inside the framework.

Under the hood, Molt runs a single trainable actor (plus an optional critic for PPO) on FSDP2, vLLM engines for rollout, and Ray queues between them, so generation, training, and weight sync overlap instead of running in lockstep.

Scaling just requires setting some flags on the same command.

The framework also splits the model across GPUs for you, so the script that trains an 8B model on one node can also train DeepSeek-V3-scale MoE models on a cluster by raising the parallelism flags.

Here's the repo: https://github.com/NVIDIA-NeMo/labs-molt

Since the framework is new, I am still experimenting with it and will share more about my learnings soon.

To dive deeper into how top AI labs are building RL Agents in 2026, I wrote a full article about it.

It covers the evolution from RLHF to GRPO with verifiable rewards and using an LLM-as-judge reward function. This idea extends RL training to non-verifiable agentic tasks without writing custom reward functions.

Read it below.