Live and Learn for the Best