I've been doing something like this for a while, and it provides decent enough results for agents to one-shot. The key is to specify the LLM with the idea, and then get it to ask you questions until you feel enough ambiguity has been eliminated from the product spec for you to pass it to an agent.