SimuHome: A Temporal- and Environment-Aware Benchmark for Smart Home LLM Agents

TL;DR

We introduce $\textbf{SimuHome}$, a high-fidelity smart home simulator and a benchmark of 600 episodes for LLM-based smart home agents. Existing smart home benchmarks treat the home as a static system, neither simulating how device operations affect environmental variables over time nor supporting workflow scheduling of device commands.

Opening excerpt from the authors’ abstract. source