When Tracking Agents Isn't Enough