Most aren't actually. Pharma (and now GE) aside, it was very uncommon in Boston for a variety of historical reasons. (Companies that were in Boston/Cambridge were actually moving out over time until quite recently.) Nor was the city of San Francisco (as opposed to the South Bay) a tech employment center until the past few years when it started to change.

But certain urban centers have become popular living locations for (mostly) people right out of school. They may also be near customers, especially in finance and related industries. (Even when engineering offices were in the suburbs/exurbs, it's always been fairly common for enterprise tech companies to have downtown sales and consulting offices.)