I think the trany and axles will work ok in theory ..........but , you have another problem . The IRS type 3 torsion housing does not provide any rear mounting for the trans . No trans frame horns like swingaxle cars have . IRS cars have the eng & trans hanging from the nose cone at the front and the body at the rear. Your 64 body will not have those mounting points at the rear . This will be a challenge for sure .
allthough , if just want to use the pan because it's good you could use your old swing axle housing on the new pan and keep it a swing axle car.