Sharks CEO Smit, who skippered the Springboks to their World Cup win under White six years ago, confirmed the appointment in a statement, ending speculation of where the 50-year-old would continue his career having surprisingly quit his role as Brumbies coach last month.
"We are immensely happy to confirm the appointment of Jake White as our new director of rugby," Smit said. "We formed a formidable relationship during my playing career and I am delighted to re-establish that partnership, which is in line with our vision to take the Sharks to the next level.
"Jake is an experienced coach whose track record speaks volumes about his ability to rally his team and get the best out of them. We are pleased that he has decided to join the Sharks."
White takes over from Brendan Venter, who was appointed in June this year following the axing of New Zealand-born coach John Plumtree after a disappointing Super Rugby campaign.
Venter remains director of rugby at English side Saracens having performed a dual role, though Smit hopes to keep him involved in some capacity with the Sharks.
"Brendan Venter's commitment to me was to drastically change our players' environment which we think he has done with extreme success," Smit continued.
"Brendan has no written contract with the Sharks, which is a testament to his character and his positive motives in coaching us and we will do whatever we can to keep him involved with us for as long as possible, in any capacity."
White led the Canberra-based Brumbies to the Super Rugby final in 2013, where they lost 27-22 to New Zealand's Chiefs, but quit last month citing the need to be closer to his family based in South Africa.