Since you mentioned "ideal", let me give you an example of "ideal". ＂中国梦＂, literally translated as "the Chinese dream" and a phrase coined by CCP, describes an "ideal" that fits your description.
But no, "beautiful" in this context just means prettier physical outlook of buildings. You can't over-interpret it (at least in Chinese).
I think you only understood part of what it tries to convey.
Good aesthetics, yes, part of '美丽'. But I won't say '美丽' stops short at simply say 'we will build our cities to look pleasing', at least I won't expect the Constitution to be so pragmatic, because it serves as an guide on the ideology (sort of).
'美丽' is used to describe 'ecological civilization', not to describe beautiful cities.
It is used against the backdrop that pollution (of water, air, soil, etc.) is very serious across China, and there are many complaints from ordinary people.
You can see the meaning of '美丽' from this sentence '生态环境根本好转，美丽中国目标基本实现' (the ecological environment has fundamentally improved, and the goal of a beautiful China has basically been achieved).
This is the speech Xi delivered at the 19th National Congress of the Communist Party of China. It contains this sentence. If you skim the speech, you will find multiple mentions of '美丽', all of them environment related.
The other four words are related to other aspects of policies of China.
富强(prosperous) is related to economy.
民主(democratic) is related to politics.
文明(culturally advanced) is related to culture.
和谐(harmonious) which was added to the constitution by Xi's predecessor Hu Jintao, is related to society.
And 美丽(beautiful) is related to ecology or environment.
------
富强, 民主, 文明 and 和谐 are the first words of the 'core socialist values' which contain 12 such words and which were laid out at the 18th National Congress of the Communist Party of China, when Hu stepped down and Xi become the paramount leader of Communist Party of China.
'Core socialist values' are taught to students from kindergarten to college. And these words of 'core socialist values' are plastered over walls, buildings and flyovers at every cities in China.
As you can see, the paragraph on "beautiful" is just a tiny part of the constitution and the word "beautiful" only appears once throughout the entire constitution. So it is probably not something very "guiding" and "central".
Also, the constitution touches on a lot of aspects of the society, so I wouldn't be surprised if it is amended to include a small mention of the "look" of the physical landscape.
I would say that the word is added just to make sure that the party keep environmental concerns and "good outlook" in mind while working on all the other tasks.
If the entire Constitution is about concrete matters comparable to 'building good-looking cities', and 'building good-looking cities' is certainly one such matter because it is emphasized in the concluding remark, then why would the word appear only once?
I don't believe abruptly inserting a term that's never mentioned is good writing style.
A plausible way to understand it is that, the concluding remark is a summarizing call-to-action, laying out the spiritual and ideological framework for the rest to act upon. It certainly needs to be abstract, just as the four words preceding 美丽.
But no, "beautiful" in this context just means prettier physical outlook of buildings. You can't over-interpret it (at least in Chinese).