Opposition MPs said his tardiness showed disrespect for his office and boycotted a meeting of the budget committee for five hours in protest.
They have been highly critical of Mr Sakurada after a series of gaffes.
Last week, he said he felt let down after Olympic swimming hopeful Rikako Ikee was diagnosed with leukaemia.
"She is a potential gold medallist [at the Tokyo 2020 Games], an athlete in whom we have great expectations. I'm really disappointed," he was quoted as saying. He was then forced to apologise after being criticised.
In 2016, he also came under fire for describing so-called comfort women forced to provide sexual services to Japanese war-time troops as "professional prostitutes".
And last year, Mr Sakurada, who is also the cyber-security minister, said he had never used a computer, adding he had always delegated the work to his subordinates.
The opposition has repeatedly called for his resignation.