The 50-year-old man was found lying on the ground under a scorching sun on Thursday afternoon at the site where he had been laying electric cables.
He was rushed to hospital but later confirmed dead, according to Tokyo 2020 organisers on Friday.
Organisers said the cause of death has not been officially established but several local media cited police as saying they suspect heatstroke. Police were unable to confirm the reports immediately.
The death is the third related to Tokyo Olympics construction works after one suicide from overwork stress and another killed in an accident.
Tokyo's hot and humid summer weather is a major concern for Olympic organisers, particularly after a heatwave that engulfed the city last year.
After a long rainy season this year, the Japanese capital has again been hit by a deadly heatwave. Last week more than 18,000 people had been hospitalised nationwide due to the weather with 57 dying.
Organisers have come up with a variety of cooling methods ranging from misting sprays and fans, to potted flower fences that organisers say help "psychologically" cool those around.
They have also moved up the starting times for some sports, including the marathon, over fears for athletes' wellbeing.