> so even cutting that cost in half doesn't save much on the whole project.

Houses are usually a leveraged investment. If you put down 5%, and prefab saves you 5%, you have massively reduced your risk of default during a downturn.

And if you expect it to take 20 years to pay off your mortgage, and framing saves 5%, that’s one year saved, which is massive IMHO.

Obviously a lot of assumptions to the above e.g. everyone else doesn’t get the 5% savings, and everyone doesn’t just bid up new house prices by 5% (because they are bounded by their mortgage which is related to their earnings, not costs).

These are fair points, and if a buyer had 100% confidence in prefab, your argument certainly makes financial sense. But for most consumers prefab is a novel approach, and therefore likely to be perceived as risky. So that savings might need to be greater to get them to try something new. Given the failure of prefab to make a big dent in the market, that seems to be exactly what's happening.

I am currently in the process of buying prefab house and after visiting couple of local factories it seems like most of them is doing just framing and competing on price only.

We are going with company that actually assemble full house inside and bring it to site fully equipped: electrical, plumbing etc.

What was the breakdown for prefab direct cost versus transportation cost (if you don't mind me being nosey)?

Transportation should be around 5-8%.