Goss is green jersey holder Peter Sagan's biggest rival in the points competition, and got the better of the Slovakian at the intermediate sprint to begin closing his 27-point overnight deficit.
After some solid work by his Orica-GreenEdge team, Goss beat Sagan at the finish again, but was later sanctioned for coming out of his sprint line and blocking Sagan as the Liquigas rider attempted to pass.
Goss's move, which came around eight minutes after Britain's David Millar had claimed the stage win, prompted Sagan to gesture angrily in protest.
It was later upheld by race director Jean-Francois Pescheux, who explained: "He didn't stay in his line. He changed his line then made quite a big mistake, it's clear to see.
"He has been relegated to last place on the stage."
The sanction means Goss keeps the points he won on the day but is hit with a 30-point penalty.
He now has 198 points, 56 shy of Sagan's 254.
Even before the ruling, Goss appeared to be anticipating a penalty.
"Whatever happens, happens," said the Tasmanian.
"Sometimes it just depends on who's making the decision. Some commissaires are more lenient than others.
"It doesn't really matter. The rules are set out from the start. I'm not going to worry about it, I'm just going to focus on tomorrow."
Sagan later said: "If he had beaten me normally, I would have accepted it. I wouldn't have been happy, but I would have accepted it.
"If he's lost points, that's his fault not mine."