David Ngog, who scored both goals in the first leg last week, wrapped matters up when he headed Liverpool ahead after 22 minutes after a perfect pass from Cole.
The five-time European champions doubled their lead after 40 minutes when Steven Gerrard cooly wrong-footed the Macedonian side's goalkeeper Martin Bogiatinov from the spot after Fernando Lopes hauled down Ngog.
England midfielder Cole, who moved to Anfield from Chelsea last month, almost scored seconds after halftime when his far post cross hit the bar.
The visitors were totally outplayed and only escaped a much heavier defeat because of some wayward Liverpool shooting, the only minor niggle for new coach Roy Hodgson in his first match in charge at Anfield.