Wild fluctuations in idle below 1300 rpm

Finished building up a 3S-GTE engine for an MR2 but could not get it to idle properly below 1300rpm. Anything lower would gradually cause the rpm to fluctuate more and more the more I close up the ISC or idle adjustment screw and would eventually fluctuates to a stall. ISC is on open loop and I have went as far as plugging the ISC port up to control idle with the adjustment screw only so it is not the ISC settings that are causing the fluctuations.

Aftermarket cams were installed on the car and degreed while building up the engine. These were Kelford 264 cams which isn't that aggressive. However, the final adjustment on the cam gears were a little worrisome as the adjustment were pretty must at the end of the adjustment scale on the exhaust gear with 11 degrees of retard but I did measure it two more times while building up the engine to make sure it is correct. I know aggressive cams will require a higher idle due to note being able to pull sufficient vacuum but 262 cams are far from aggressive and looking at my MAP readout, vacuum is pulling strong. I have another 3S-GTE engine with the same cam that idles just fine at 900 to 1000 rpm.

Lambda looked stable enough to me and would be fine tuned once the idle problem is solved and I have already flattened the ignition map a little to make sure its fluctuation between the rpms isn't causing the problem.

Right now, I am truly stuck on what's causing the crazy fluctuations in rpm and cannot progress any further with the tune.

Attached is the latest map used and a log from startup where the rpm was stable and high until the engine warmed and the ISC valve started closing up and the fluctuations begins. Manually pushing the ISC% back up rises and stabilises the rpm.