graph-reasoner
Structured (graph) rewards for logical reasoning.
graph-reasoner is two things at once: a verifiers v1 environment for logical entailment, and — as this site — the paper describing it. They are written together, so the science explains the code and the code illustrates the science.
The idea
Outcome-only RL (GRPO / R1-style) gives one sparse reward bit per rollout. But when a task's reasoning trace has a recoverable graph structure, the ground-truth graph can be derived automatically — by forward chaining — and we can reward structural coverage of it. That buys PRM-like reward density with no learned scorer and no human labels. Whether that density helps is the research bet.
The task
Given a theory — facts plus function-free Horn-clause rules over entities — and a query atom, the model decides entailment and emits the derivation graph. We instantiate this on RuleTaker / ProofWriter. Ground truth comes from forward chaining (Datalog least-model semantics); the reward is a graded coverage score plus an answer↔graph consistency check. Concretely, the coverage term is the node-classification \(F_1\) of the predicted graph \(\hat{G}\) against the closest gold derivation \(G^\star\):
Status
Early. The package is scaffolded and the data pipeline is in progress. This page grows into the full paper as the work lands — read it as a living document, not a finished one.