However, their late pressure failed to produce an end result and the home side drew level on six points with both Scotland and Slovakia in Group 3.
Stark made one change from the team that started in the 2-1 win over Slovakia in Paisley on Thursday with Hearts midfielder Jason Holt replacing MacLeod at the Mikheil Meskhi Stadium.
Scotland started well and came close to an eighth-minute opener when Dylan McGeouch hit the post after Stuart Armstrong had seen a shot parried following good work from Stevie May.
However, Georgia then had some good spells of possession without testing Archer.
Armstrong and May combined again midway through the half with the Dundee United midfielder this time playing in the St Johnstone striker, but May could not get enough power in his shot and Giorgi Nadiradze saved.
Scotland then came under pressure and Archer made an excellent save to stop Jigauri converting at the near post.
Giorgi Kvilitaia blasted over after a corner before Georgia took the lead in the 39th minute when Kazaishvili fired low into the corner of the net.
Archer made second-half saves from Kazaishvili and Chanturia but the Tottenham goalkeeper was beaten again in the 70th minute when Jigauri netted from 20 yards following good work from Kazaishvili.
Matters got worse for Scotland when Celtic defender Stuart Findlay was ruled to have used his arm in a penalty box challenge and the referee pointed to the spot.
Chanturia converted the initial penalty but a re-take was ordered and Archer dived low to stop the second effort.
That sparked a Scotland fightback and in-form MacLeod pulled one back two minutes later when he fired home from 25 yards.
The Scots pushed for an equaliser but Aberdeen full-back Clark Robertson had a close-range shot saved and Hearts defender Jordan McGhee headed over from an injury-time corner.
The defeat is Scotland's second in four qualifiers following their 4-0 loss to Holland last month.