John Swinney told Nicola Sturgeon he was "nervous" about her push for a fresh referendum after Brexit.